To optimize the trade off between the performance and lightweight design of battery pack for new energy vehicles, this paper proposes a multi-objective optimization algorithm (FMO-RL) based on deep reinforcement learning (DRL). Built upon the NSGA-II/III framework, FMO-RL integrates an innovative DRL agent coupled with adaptive parameter adjustment and adaptive multi-operator selection mechanisms, enabling dynamic optimization and regulation of multi-objective optimization model parameters. Experimental results on the standard DTLZ1–DTLZ3 benchmark suites show that FMO-RL achieves an average hypervolume (HV) loss of less than 2%, while reducing the inverted generational distance (IGD) and generational distance (GD) by 30%–75% compared with classical evolutionary algorithms. When applied to the multi-objective lightweight optimization of a vehicle battery pack, FMO-RL reduces the total weight of 9.5 kg while fully satisfied all performance constraints, including battery pack strength, modal characteristics, and electrothermal performance.