Reproducibility is a crucial aspect of neural network training, but it remains challenging due to the stochastic nature of the optimization during the training process. This paper examines the effects of two weight optimization techniques: weight selection optimization and traditional weight update optimization, on the run-to-run variability of neural network performance. Our extensive experiments across various convolutional layer sizes reveal that weight selection optimization reduces variability by approximately \(25\%\) to \(40\%\) , depending on the model, compared to traditional training methods. However, while variability is reduced, model accuracy remains unchanged, staying on par with conventional training. These findings provide valuable insights into enhancing the stability and reproducibility of neural network models without compromising accuracy.