This paper proposes a hybrid approach for student engagement detection. The main contribution of this work is the combination of handcrafted features and transformers to enhance model performance, even when trained on a relatively small dataset. This is achieved by using Histogram of Oriented Gradients (HOG) descriptors to extract features from students’ faces and apply transformers to learn the relationships between these features and student engagement. To validate the accuracy of the proposed model two benchmark datasets, FER13 and CK+, and a custom dataset comprising over 700 frames were tested. The model was also compared to several deep learning models, including EfficientNet, Inception-V3, ResNet, and ViT. It is found that the proposed model outperforms state-of-the-art methods by achieving an accuracy that exceeded 76%. This approach is accurate, reliable, real-time, and adaptable to various contexts.