ProdNet: A Lightweight Network for Fast Discovery of Matrix Multiplication Algorithms
摘要
Matrix multiplication is a fundamental operation with significant applications across diverse fields, such as physics, electronics, and artificial intelligence. Traditional implementations of this operation exhibit cubic time complexity, which presents computational challenges, particularly in deep learning scenarios that necessitate large-scale matrix computations. In this study, we introduce ProdNet, a lightweight neural network model designed to autonomously discover efficient matrix multiplication algorithms without the need for extensive computational resources or prior knowledge of existing methods. Our approach seeks to alleviate complexity by minimizing the number of multiplicative operations involved. To achieve this, we utilize a combination of Mean Squared Error (MSE) and a regularization function, targeting weight values to be constrained to 0, 1, or –1. We emphasize the effectiveness of our regularization techniques in accelerating the algorithm discovery process. Furthermore, we demonstrate that ProdNet successfully identifies multiplication algorithms for 2 \(\times \) 2 and 3 \(\times \) 3 matrices, achieving this in a limited number of iterations (starting from 500 iterations for \(2 \times 2\) and 8800 iterations for \(3 \times 3\) )