In modern data-driven applications, machine learning models deployed in dynamic environments face significant challenges due to data drift, which refers to changes in data distribution over time. Unaddressed data drift can severely degrade model performance, especially in fields such as healthcare, finance, and industrial automation. Traditional drift detection methods often rely on supervised approaches requiring labeled data, making them costly and impractical for real-world applications. This paper explores the potential of Large Language Models (LLMs) to detect data drift in an zero-shot manner, leveraging their contextual understanding to identify both statistical and semantic anomalies without relying on labeled instances or predefined drift metrics. We investigate three critical aspects influencing the effectiveness of LLM-based drift detection: (1) the benefit of monitoring raw data features directly, (2) the impact of varying input token lengths, and (3) the model’s sensitivity to different types of distributional shifts (white noise, gradual degradation, and sudden setting changes). Experiments conducted on sensor data from a counter pressure casting (CPC) process reveal that LLMs effectively identify abrupt drift but show reduced sensitivity to gradual shifts and negligible responsiveness to random noise. Our findings provide insights into the advantages and limitations of employing LLMs for zero-shot drift detection, suggesting their high potential in real-world monitoring applications when combined with optimized input handling strategies.