Distributed Data-parallel (DDP) computing allows training a model cooperatively with multiple peer processes running in parallel, avoiding a fully centralized architecture. This approach allows distributing the workload across multiple nodes, which is beneficial in contexts with limited computational and memory resources. High-performance tools support distributed training through data parallelization and communication primitives for efficient information exchange. One DDP approach is Synchronous Data Parallelism, where the model is replicated across multiple devices. In parallel mode, each device trains a local model on its data partition and independently computes the gradients. During training, the gradients of all devices are synchronized and aggregated to update the parameters of the global model. This work explores various gradient reduction operations, including SUM, MEAN, MAX, MIN, and a novel operation called BATMAX, to assess their impact on the effectiveness of DDP training using two benchmark datasets: MNIST and CIFAR-10. The analysis verifies whether the data-parallel approach, combined with a specific reduction operation, can achieve performance comparable to the sequential context in terms of both accuracy and execution time. Experimental results demonstrate that, in a multi-process environment, the choice of gradient reduction operation significantly affects convergence. Moreover, convergence speed is influenced by data loading, synchronization, and the operation’s computational complexity.