The effectiveness of optimizing complex problems is closely linked to the configuration of parameters in search algorithms, especially when considering metaheuristic optimization models. Although various automated methods for algorithm configuration have been proposed to alleviate users from manually tuning parameters, there is still unexplored potential in dynamically adjusting certain algorithm parameters during execution, which can lead to enhanced performance. The main objective is to comparatively analyze the effectiveness of manual parameter tuning compared to a dynamic online configuration approach based on reinforcement learning. To this end, the State-Action-Reward-State-Action (SARSA) algorithm is adapted to adjust the parameters of a genetic algorithm, namely population size, crossover rate, mutation rate, and number of generations. Tests are conducted with these two methods on benchmark functions commonly used in the literature. Additionally, the proposed model has been evaluated in a practical problem of optimizing energy trading portfolios in the electricity market. Results indicate that the reinforcement learning-based algorithm tends to achieve seemingly better results than manual configuration, while maintaining very similar execution times. This result suggests that online parameter tuning approaches may be more effective and offer a viable alternative for optimization in metaheuristic algorithms.