<p>Twin support vector machine (TSVM) with a hinge loss function is a prominent and effective machine learning technique for classification problems. However, the use of the hinge loss function makes it sensitive to noise and outliers, and also its non-differentiable nature limits its applicability to gradient-based optimization methods. Additionally, TSVM employs the empirical risk minimization principle only, which may lead to overfitting issues. To address these challenges, we propose a novel approach called a robust twin support vector machine with huberized hinge loss function (HTSVM). This loss function is smooth and less sensitive to outliers which enhances the models robustness and optimize its performance. The smoothness of this loss function facilitates the use of the proximal gradient method to solve the proposed HTSVM model. Moreover, the elastic net regularization technique has been implemented to incorporate the structural risk minimization principle, which prevents the model from overfitting. This regularization technique provides a more flexible and robust approach for addressing multi-collinearity and feature selection in high-dimensional datasets. Extensive experiments are conducted on thirteen benchmark datasets, evaluating average accuracy, sensitivity, specificity, F-measure, and Matthews correlation coefficient. The results clearly indicate that the proposed technique outperforms existing state-of-art methods by achieving an average accuracy of 82.97%. Additionally, statistical analyses utilizing the Friedman and Nemenyi tests reinforce that the established HTSVM method offers superior generalization performance. Furthermore, to demonstrate the effectiveness of the proposed HTSVM method, this model has been applied to the handwritten digit dataset, showcasing its potential to address real-world problems.</p>