Convolutional Neural Networks (CNNs) offer the potential to significantly enhance the early detection of GastroIntestinal Cancer (GIC), because it is a disorder that impacts new cases and deaths every day. For this reason, we proposed an automated method based on multiple stages to detect GIC. Firstly, we apply the Encoder-Decoder Network (EDN) to detect the Region Of Interest (ROI) of polyps in preprocessing. Secondly, we compare many pretrained models such as VGG16, VGG19, ResNet50, and InceptionV3 to choose the best for feature extraction. Lastly, a classifier technique is a Support Vector Machine (SVM). For this purpose, we employ five datasets with various image types. According to our results, pretrained models perform well with SVM, reaching the maximum accuracy of 98.90%. Our suggested approach yields the most accurate results for the precise identification of gastrointestinal cancer.