Deep Learning (DL) models, particularly Convolutional Neural Networks (CNNs), are extensively used in computer vision tasks. However, the inherent “black-box” nature of these models poses challenges in understanding their decision-making processes. This study presents a comparative analysis of CNN models with various merging layers for multi-class classification using a public dataset, with the comparison based on evaluation metrics, interpretability plots such as Gradient-weighted Class Activation Mapping (Grad-CAM), Local Interpretable Model-agnostic Explanations (LIME), and feature maps. Leveraging merging layers available in the Keras TensorFlow library, the research investigates the impact of different merging operations on both model performance and interpretability. The results indicate that specific merging techniques, such as minimum and maximum operations, not only enhance classification accuracy but also improve model transparency. These findings offer valuable insights into CNN’s architectural design, contributing to the development of more reliable and interpretable DL models.