In recent years, the development of large-scale datasets and computational power has skyrocketed, significantly advancing the field of handwriting recognition of historical artefacts. Mainstream approaches in the field have historically relied on convolutional recurrent neural networks. However, these deep learning models encounter numerous challenges when dealing with long text images, such as gradient vanishing and gradient explosion. Another drawback is the large number of parameters of these models, which requires a large amount of computation. In response to these limitations, this paper introduces a novel Handwritten Text Recognition (HTR) architecture that integrates a gated convolutional neural network (Gated-CNN) with an ALBERT self attention mechanism. This proposed architecture is built upon the Spatio-Temporal Attention Network (STAN) architecture and is designed to enhance recognition efficiency while reducing the computational burden. Experimental validation conducted across various datasets demonstrates that this model can surpass the performance of other mainstream models. Notably, it achieves this superior recognition capability with fewer convolutional layers and a reduced parameter count, thereby addressing the critical drawbacks of existing approaches.