This work presents a comprehensive study of the application of multi-agent reinforcement learning (MARL) based on deep Q-networks (DQN), aiming to enhance the cooperation and coordination of multiple agents in complex environments. The core problem addressed is the multi-agent traveling salesman problem, i.e. the effective collaboration and target-reaching by multiple agents. This challenge is inherently present in a variety of applications, including inspection and maintenance tasks based on autonomous aerial robotic systems. The research delves into the intricacies of multi-agent systems, emphasizing the dynamic interaction between agents and the environment. Although the proposed algorithm is developed and trained in a planar space, its principles and methodologies are easily extendable to any operational environment. The proposed approach is capable of training on a number of different scenarios, using various numbers of agents and targets, regardless of their initial positions, making it highly adaptable to scenarios like aerial inspections, where the number of agents, the number of inspection sites and their locations are unknown a priori. Furthermore, the work outlines how the enhanced DQN algorithm can be tailored for scenarios with numerous autonomous agents and targets, ensuring efficient path planning and target acquisition in constrained and unstructured spaces. This work also provides a solid foundation for future research in integrating advanced reinforcement learning techniques into the design and operational strategies for solving tasks, which can be described by the multi-agent traveling salesman problem, including inspection and maintenance with an autonomous multi-agent system.