Aiming to optimize the scheduling optimization problem of the air conditioner direct shipment assembly line at Company H, this paper proposes a deep reinforcement learning method based on prioritized experience replay with a double deep Q-network (DDQN). First, the assembly line is abstracted as a flexible job shop scheduling problem (FJSSP) with 10 workpieces × 6 machines, and a mathematical model is established with the optimization objective of minimizing the total delay time. The algorithm is designed to describe the scheduling environment through 7-dimensional state features (including machine utilization, process completion rate, and ratio of delayed workpieces, etc.), to construct the action space by incorporating five heuristic rules, and to design a composite reward function that combines the variation of the delay time with the machine utilization. A Boltzmann exploration strategy and a fully connected neural network (7–30 × 5–6 structure) are used for Q-approximation. Experiments show that the algorithm optimizes the dragging time from 198 to 88 on a 10 × 6 historical dataset, which is close to the effect of the genetic algorithm; on a random dataset simulating an air-conditioning production line, the dragging time is reduced from 88 to −15, which verifies the effectiveness and generalization ability of the method.