In this study, we present an innovative method for exploring the application of transfer learning using the VGG16 model pre-trained on ImageNet for classifying chest X-ray images into two classes: normal and pneumonia. The TensorFlow framework is utilized for implementation. We employ data augmentation techniques during training to enhance the model’s generalization ability. The performance of the model is evaluated on a separate test set, demonstrating promising accuracy and efficacy in detecting pneumonia from chest X-ray. Notably, our approach involves training the model entirely from the ground up, ensuring a unique and original contribution to the field. In contrast to existing methods that heavily depend on manual techniques or transfer learning strategies, our approach demonstrates remarkable classification performance. Instead of leveraging pre-existing knowledge or models, we developed a CNN model from the ground up, allowing it to effectively identify pneumonia by extracting relevant features from specific chest X-ray images and making accurate classifications. This model may be used to address reliability and interpretability problems that frequently occur while working with medical images. Compared to other deep learning classification issues with significant picture repositories, collecting a large number of pneumonia datasets can be challenging for this classification work. To enhance the CNN model’s classification and validation performance and obtain excellent validation accuracy, we employed several data augmentation techniques.