The present paper sets out a training environment for reinforcement learning (RL) models in the context of cybersecurity management. The environment is designed to replicate changes in service status in the event of attacks by an external entity, thus facilitating the training of RL models to support cybersecurity management processes. Two RL algorithms, Proximal Policy Optimization (PPO) and Deep Q Network (DQN), were evaluated in the developed environment. The results demonstrated that the DQN agent exhibited a higher detection rate and required a smaller average number of actions to detect the attacker. In contrast, the PPO agent has been shown to be more effective in limiting the damage caused to the network. The development of this training environment validates the proposal to use RL-based agents that act without human intervention for cybersecurity management and opens up new possibilities for training models capable of protecting the network.