Convolutional Neural Networks (CNNs) are crucial for processing multimedia data. Their growth has been fueled by advancements in computing platforms, e.g. GPUs and TPUs. With massively parallel computing capabilities, GPUs can significantly speed up the training and inference of CNNs. The advent of CUDA, a parallel computing and programming model, has also eased efforts to parallelize these computational tasks, primarily attributed to the convolutional layers, on GPUs. However, the current implementation of convolution algorithms still has many limitations, lowering the performance of the convolutional layers. In this paper, we propose a method to optimize Winograd-based convolution on GPUs to speed up convolution operations. Experiments on a commercial GPU demonstrate that the proposed method remarkably outperforms two state-of-the-art algorithms in the cuDNN library, GEMM and Winograd. In addition, the proposed method also brings significant benefits in terms of reducing the required memory usage. These results have a positive impact on accelerating CNNs.