The optimization of convolutional neural network (CNN) training poses a formidable challenge due to its high-dimensional and non-convex nature. This issue is particularly pronounced when setting parametric learning rates lacks confidence, leading to inefficiencies in the training process. Prior research has explored the integration of Newton’s methods to address such challenges encountered in deep neural network training. However, the application of Newton’s methods to CNNs entails intricate operations, especially in computing the requisite Hessian matrix for second-order methods. This complexity is further compounded when employing finite difference methods, particularly with image data. To tackle these computational challenges, researchers often use a variant of Newton’s method for CNNs that incorporates a sub-sampling strategy for the Hessian matrix. However, in this investigation, we deviate from conventional approaches by opting for the utilization of complete data sets instead of handling partial data subsets at each iteration. Furthermore, we enhance computational efficiency by introducing parallel processing techniques in lieu of serial processing during mini-batch computations. Our experimental findings underscore the efficacy of parallel processing, as it significantly surpasses the time efficiency achieved by previous methodologies. We demonstrate improved convergence rates and overall performance in CNN training using our proposed technique.