This study introduces an advanced technique for classifying biomedical images, with a focus on accurately identifying polyps in frames captured from video endoscopies. Our approach utilizes tailored image descriptors designed to enhance the precision and reliability of polyp detection. By incorporating these descriptors, we tackle the distinct challenges presented by biomedical image analysis, especially within open-access datasets. Notably, our method achieves superior classification performance while maintaining a lightweight structure, offering a streamlined alternative to existing resource-intensive deep neural network models. Evaluation metrics underscore the effectiveness of our approach, with a precision of 0.983, recall of 0.969, and F1-score of 0.976. This proposed technique holds promising implications for medical diagnostics, representing a valuable step toward more efficient and accessible tools for automated analysis in medical imaging, without heavy computational demands.