Given how complicated and volatile the stock market is, predicting price is a challenging task. Researchers and investors are investigating ways to predict the stock market with high accuracy so that maximum profit can be obtained. In this paper, we introduce a model aimed at predicting stock prices that uses EMA-BiLSTM model that provides high accuracy with optimum error in forecasting stock price in comparison with long short-term memory(LSTM) and CNN-based LSTM model. The proposed deep recurrent neural architecture (DRNA) has the capacity to process in both to and fro direction thus improving prediction performance in stock price forecasting. In this paper we have compared our model with conventional LSTM and LSTM with CNN layer both theoretical and numerical values obtained on United States Steel Corporation (USS) stock prices. The suggested approach processes input sequences in both forward and backward directions, also applying an exponential smoothing function to the internal memory states. This helps the model better understand and remember long-term patterns in the data. Different variants of LSTM are compared on the basis of MAE, R \(^2\) , and MSE. The performance R \(^2\) metric of the proposed model is 0.8961 as compared to 0.7219, 0.3901 for simple BiLSTM and CNN-LSTM, respectively.