The advancement of autonomous driving technology has conditioned the need for robust and efficient automated systems to ensure safe navigation in different traffic situations. Despite the existence of models that incorporate a variety of risks, there is still a possibility of conditions that cause discrepancies, which could, in turn, produce hazardous situations. This research paper proposes an approach with multi-objective optimization problem that integrates a critical safety metric into a reinforcement learning (RL) algorithm to improve autonomous vehicle (AV) safety. The aim is to develop an adaptive AV decision-making framework that includes a control strategy enabling the agent to improve its driving performance through interaction with simulated urban environments. The primary focus is on the Advanced Longitudinal Control system, particularly in scenarios involving critical vehicle maneuvers, such as cut-in in dense traffic flow. The learning algorithm is implemented using MATLAB-Simulink® software, providing a robust simulation environment. This approach allows for an in-depth understanding of urban traffic challenges using high-fidelity simulations for the RL model's training.