Medical images play a crucial role in diagnosis and anatomical studies. Convolutional Neural Networks (CNNs), especially when utilizing transfer learning, have become essential for image classification in medical applications. However, the interpretability of CNNs remains a significant challenge. This paper proposes a methodology based on feature visualization and heatmaps to interpret the aspects of input images influencing label assignments. It includes ad-hoc CNNs and those based on the ImageNet dataset, including VGG19, ResNet, Inceptionv3, and MobileNet. Various classification schemes, training approaches, and data augmentation methods are explored. Feature visualization is performed on pretrained CNNs, so input patterns maximizing each output neuron response are identified. Heatmaps, which identify the regions of the image responsible for determining the classification, are generated. The proposed approach is tested on diverse medical image datasets, and a Python toolbox based on Keras is available on GitHub. Results demonstrate the reliability of the proposed approach in interpreting patterns and associating them with labels in medical images.