Hyperparameter selection, an important and challenging problem in machine learning, is particularly crucial for achieving optimal performance. Least Squares Support Vector Regression (LSSVR) combines the powerful capabilities of Support Vector Machines (SVM) in machine learning with the simplicity of the Least Squares method. In this paper, we propose a new hyperparameter selection method for LSSVR by constructing a novel optimization problem, which is then solved using the gradient descent algorithm. By comparing the performance of the grid algorithm and several heuristic algorithms in terms of mean squared error (MSE), the experimental results demonstrate that our method can select more suitable parameters compared to other approaches, ultimately leading to a smaller MSE value.