This paper proposes an efficient emotion recognition method, achieving classification accuracies of 98.57% on JAFFE, 98.06% on KDEF, 99.3% on CK +, and 99.72% on Oulu-CASIA. The proposed approach utilizes two lightweight convolutional neural networks (CNN), ShuffleNetV2 and MobileNetV2, for feature extraction. The methodology includes four key stages: preprocessing of face images, feature extraction, dimensionality reduction via Neighborhood components analysis (NCA) to produce three distinct feature vectors with different dimensionality, and classification based on support vector machine (SVM). In the training phase, three SVM models are trained using training sets composed of feature vectors with different dimensionalities. In the inference phase, a soft voting mechanism is employed, combining predictions from three individually trained SVM models. The results emphasize the balance between accuracy and computational efficiency, a critical factor for real-world applications requiring real-time processing. This work demonstrates that high performance in emotion recognition can be attained without reliance on overly large architectures, thereby enhancing accessibility and ease of implementation.