Colorectal cancer is the third most common cancer worldwide, with a high mortality rate if left untreated. Early detection and characterization of precancerous polyps are crucial for prevention, and automated tools can aid in reducing manual labor and improving accuracy. This study presents a deep learning approach for classifying Hamartomatous (benign) and Adenomatous (pre-cancerous) polyps using transfer learning with ResNet 101 and EfficientNet B2 models. Detail ehe experimentation was conducted on a custom dataset consisting of 1706 annotated polyp images, derived from colonoscopy procedures on 45 patients at IILDS India, containing 49 unique polyp cases. Our results show that both models are effective in characterizing polyps, with performance equivalent to or exceeding manual characterization by doctors. ResNet 101 provided 98% sensitivity and 85.26% accuracy for the best experiment while EfficientNet B2 provided 99.47% sensitivity and 82.05% accuracy. This automated tool has the potential to assist in biomedical imaging for early detection and prevention of colorectal cancer.