Gastrointestinal diseases such as colorectal cancer and inflammatory bowel disease are severe global health problems that need to be diagnosed early and precisely to treat them. The rising incidence of such diseases necessitates the development of new diagnosis methods to enhance patient outcomes. These conditions affect patients’ quality of life badly and impose a heavy health system burden. Deep learning architectures are employed for multi-class classification of gastrointestinal diseases from endoscopic images in this study. The model is experimented with VGG16, ResNet-50, and a hybrid model that integrates VGG16 and Transformer. Preprocessing tasks such as augmentation and normalization will be carried out on the 4,000 images of the Kvasir dataset. For making the model robust and generalizable, certain data augmentation techniques such as rotation, flipping, and scaling are carried out. Experimental results indicate that the best accuracy obtained using this hybrid model is 81% on the best-performing configuration, convincingly outperforming ResNet-50 and VGG16. These results demonstrate the huge potential of the hybrid of convolution and attention mechanisms toward promoting the utilization of AI-aided GI disease detection.