Medical diagnosis classification is a difficult process with a high error rate. The most popular and extensively used type of clinical examination for pulmonary modules is the Chest X-ray film because medical imaging plays a significant role in the total diagnostic procedure. However, given the high rise in infectious disease cases—a significant potential cause of diagnostic error—the number of radiologists clearly cannot keep up with this explosion. The current system classifies pulmonary images using the Inception-v3 transfer learning method. It first enhanced the pulmonary image data, then utilized to improved Inception-v3 method based on transfer learning to automatically extract features, and finally classified the pulmonary images using various classifiers (Softmax, Logistic, SVM). By employing ensemble learning and studying suitable neural network selection, the proposed system can improve its performance in classifying pulmonary images. Compared to other cutting-edge techniques, the ensemble methodology outperforms them on benchmark datasets.