Detecting and recognizing traffic signs presents major challenges for autonomous driving systems. These challenges stem from varying object sizes, environmental factors, and the need for real-time processing. Traditional models that depend on features often struggle to balance accuracy with processing speed, especially when it comes to detecting signs at multiple scales. To address these issues, this paper introduces a better feature extraction framework that combines the histogram of oriented gradients (HOG) in the HSI color space with local self-similarity (LSS) descriptors. This improved representation works with Random Forest and Support Vector Machine (SVM) classifiers to enhance detection accuracy and classification strength. Our system operates in three main stages: image segmentation based on HSI, shape analysis using geometric moments, and final recognition with a combined HOG-LSS descriptor. Tests on benchmark datasets like GTSRB and Swedish Traffic Signs reveal that the proposed model outperforms baseline methods in accuracy and adaptability, while still being suitable for real-time use. The model achieved a recognition accuracy of 99.52%, proving its effectiveness and reliability in various challenging real-world situations.