Pneumonia has caused countless fatalities around the globe and is more frequently seen among those aged 65+ and kids under five years of age. It's a serious acute respiratory infection which needs to be treated with utmost caution. Identifying pneumonia at an early stage is key to successful treatment and can be made simpler by using Computer Aided Diagnosis (CAD) systems. Chest X-Rays (CXRs) continue to be the main imaging tool used by radiologists for the detection of pneumonia and is commonly employed in clinical settings. Clinical diagnosis is the typical way to detect pneumonia, yet a variety of Deep Learning (DL) models have been established that use CAD systems for pneumonia detection. In this paper, we employed the Inception-ResNetV2 architecture in Convolutional Neural Network (CNN) to detect pneumonia. After an image is inputted, convolutional and max pooling layers are initiated with the help of ReLU activation functions. These neurons then send the data to densely connected layers before finally activating the output neuron with a sigmoid function. This process is essentially the classification of the inputted image. Training the model boosts its accuracy while simultaneously decreasing the loss. Additionally, early stopping and data augmentation minimize over-fitting by being carried out before fitting the model. The proposed deep learning model is more efficient and yield cogent results for classifying chest X-rays to detect pneumonia. Our model achieved an accuracy rate of 98.55% on Chest X-Ray Images (Pneumonia) dataset.