Early and accurate detection of brain tumors is critical to improving treatment outcomes for affected individuals. Among diagnostic modalities, magnetic resonance imaging (MRI) has emerged as a potent tool for detecting brain tumors. Nevertheless, manual diagnosis of MRI scans requires a specialist and is error-prone, time-consuming, and reliant on radiologist experience. Consequently, automated computational methods for brain tumor diagnosis offer valuable support to radiologists and clinicians. CNNs have demonstrated remarkable performance across various image analysis tasks, particularly in medical image classification. However, the effectiveness of CNNs heavily depends on the availability of a substantial amount of annotated data for training. In the medical domain, acquiring such data is particularly challenging due to privacy concerns and expert annotation requirements. Therefore, transfer learning is used to help overcome the scarcity of labeled medical data. Transfer learning leverages the knowledge from pre-trained models on large datasets and adapts itself to the task. This approach not only mitigates the data limitations but also accelerates the convergence of the model during training. This study evaluated the performance of VGG16, VGG19, ResNet50, MobileNetV2, InceptionV3, and Xception models for diagnosing brain tumors from MRI images. We used two scenarios to train and evaluate the CNN models: One involved training the models from scratch without using the transfer learning technique, and the other used pre-training models with transfer learning from the ImageNet dataset. The performance of these models was evaluated on two medical image datasets, A and B. The performance of the models varied without transfer learning; the VGG16, VGG19, and MobileNetV2 models struggled to classify images accurately. In contrast, ResNet50, InceptionV3, and Xception demonstrated better adaptability to the complexity of the datasets. With the transfer learning scenario, the VGG16, VGG19, MobileNetV2, and ResNet50 models showed marked improvements, with ResNet50 achieving an accuracy of 96.95% on dataset B, the highest accuracy rate. Unlike other models that saw improvement, the performance of InceptionV3 and Xception models on brain tumor classification tasks was notably affected by using transfer learning.