In this study, we explore the optimization of Optical Character Recognition (OCR) model performance through a systematic comparison of various backbone architectures and comprehensive hyperparameter tuning. The research focuses on evaluating the effectiveness of pre-trained models, including ResNet50V2, ResNet152V2, InceptionV3, MobileNetV2, VGG19, and DenseNet121, in addressing character recognition tasks. Our experiments demonstrate that ResNet50V2 outperforms other models, achieving a training accuracy of 95%, validation accuracy of 96%, and testing accuracy of 92%, making it the most suitable backbone for OCR tasks. The approach involves a comprehensive analysis of hyperparameters, incorporating methods such as adjusting learning rates, freezing layers, and regulating dropout using KerasTuner. These improvements resulted in a notable decrease in validation loss and improved the model’s ability to generalize. Furthermore, the models were evaluated using important metrics such as precision, recall, and F1-score, ensuring a thorough assessment of performance across different categories. pre-trained models such as VGG19 and DenseNet121 performed competitively, with VGG19 achieving a remarkable accuracy of 96%. Furthermore, we discuss the impact of model tuning on reducing overfitting and improving predictive confidence. The findings suggest that integrating advanced backbone architectures with meticulous hyperparameter tuning can substantially improve OCR accuracy, providing practical solutions for real-world applications like document digitization, text extraction, and automated character recognition. The study contributes valuable insights into optimizing deep learning models for OCR by highlighting the importance of backbone selection, hyperparameter optimization, and evaluation metrics. Future work will focus on exploring more complex architectures and incorporating additional post-processing techniques to further refine OCR performance.