Handwritten Mathematical Expression Recognition (HMER) poses significant challenges due to the complex 2D spatial structure and variable-length symbols in mathematical notation. This paper introduces custom Convolutional Recurrent Neural Network Architecture (CRNN) without relying on standard pre-trained models like DenseNet or ResNet. This paper addresses challenge of vertical alignment issue by introducing STN(Spatial Transformer Network), Vertical Positional Encoding and MHSA(MultiHead Self Attention). These components enhance the models ability to capture spatial structures and complex symbol arrangements. The proposed model has been evaluated on standard CROHME datasets, using CROHME 2014, 2016, and 2019 for validation. The proposed model achieves expression recognition rates (Exprate) of 63.47%, 57.2%, and 54% on CROHME 2014, CROHME 2016, and CROHME 2019 respectively.