Forecasting stock prices in emerging markets presents a unique set of challenges, including but not limited to heightened volatility, structural inefficiencies, and limited data availability. To understand the same, in this study, we undertake a comparative analysis of two distinct modeling paradigms – AutoRegressive Integrated Moving Average, or ARIMA, a well-established statistical approach, and Long Short-Term Memory, or LSTM, a machine learning algorithm known for its capacity to model complex temporal dependencies. Our objective is to assess their practical applicability in real-world financial forecasting scenarios, with a particular focus on emerging market equities. Drawing on historical stock data from select emerging economies like India, we implement both models under consistent preprocessing and evaluation frameworks. Performance is assessed using RMSE, MSE, MAPE, and AIC alongside resource intensity and processing time. The results reveal that while ARIMA retains value for its interpretability and efficiency in stable conditions, LSTM consistently outperforms it in capturing non-linear patterns and adapting to abrupt market shifts traits especially pertinent in less mature financial systems. This work contributes to the applied machine learning discourse by grounding model evaluation in practical constraints and market realities. It offers actionable insights for financial analysts, data scientists, and policymakers seeking to harness machine learning for more resilient and adaptive forecasting in emerging market contexts.