This study focuses on the performance analysis of optical networks using deep reinforcement learning and routing and wavelength assignment algorithms. Deep reinforcement learning is a significant estimation tool for solving optimization problems in optical networks. The routing and wavelength assignment problem is a visual networking problem to maximize the number of optical connections. We propose a Multi-Start approach that can improve the average deep reinforcement learning performance. The optical network plays a significant role in improving deep reinforcement learning performance with high link capacities. Deep reinforcement learning algorithms can concern a Genetic algorithm for saving computational time, and also focus on deep reinforcement learning algorithms for assessing the traffic matrices and total connection requests. Numerical results show that by using our novel representation, deep reinforcement learning achieves better performance and learns how to route traffic in optical networks significantly faster, and finally perform deep reinforcement learning using a topology size and link capacity.