This paper addresses the Integrated Optimization Problem of Production Planning and Re-entrant Chip Flow Shop Scheduling (IOP_PP_RCFSS), in which raw materials (i.e., wafers) need to go through multiple processing steps to be made into finished chips, and different types of chips require varying amounts of reprocessing on the wafers. This problem usually exists in the chip manufacturing factories. Due to the NP-hard complexity of this problem, an Improved Q-learning-based Algorithm (IQA) is devised to deal with it. In the IQA, Genetic Algorithm (GA) is adopted to generate high-quality initial solution, and the designed batch splitting strategies are combined with production quantity planning to reasonably divide processing batches according to actual demand quantities. Meanwhile, Q-learning in decision-making layer is used to dynamically determine search behaviors of Variable Neighborhood Search (VNS) in search execution layer. Experimental results on benchmark instances show that the devised algorithm is competitive with state-of-the-art algorithms.