<p>Autonomous driving requires highly reliable perception systems to operate safely in complex highway traffic environments. Still, current object detection models fail when given heterogeneous sensor data, inconsistent annotations, or improper parameter optimization. This framework attempts to overcome the problem through an unprecedented multi-stage approach that integrates four different datasets: BoxCars116k, Waymo Open, PixSet, and Comma2k19, via a robust data fusion and preprocessing pipeline that aligns modalities and normalizes feature spaces. YoloV5 is used to accurately segment the objects and obstacles from captured sensor data. The centerpiece of the framework is a Hybrid Graph-Based AlexNet–Inception V3 model (HGAIV3) optimized using Bobcats–Greylag Goose Optimization (BGGO) (HGAIV3–BGGO). Moreover, Graph Convolutional Networks (GCN), AlexNet, and Inception V3 are combined by a weighted feature-fusion scheme capable of capturing spatial, contextual, and hierarchical features for better recognition of vehicles, pedestrians, and road infrastructure. Hyperparameter tuning is performed using the Hybrid BGGO algorithm, which well balances global exploration and local exploitation for fast convergence and model optimization. Experimental results demonstrate that the proposed model achieves a high accuracy of 98.74%, a recall of 96.8%, a precision of 97.5%, and an F1 score of 97.2%, alongside reduced latency of 4.021 seconds and a computation time of 27.32 seconds for 30 frames. The results bring out the framework’s novel aspect in creating a unified dataset across multiple modalities, combining complementary deep learning (DL) architectures, and proposing a more effective optimization mechanism to ultimately improve the making of safer and more adaptable autonomous driving systems.</p>