A nuclear power unit chiller compressor frequently experiences gear tooth failure during operation, directly affecting the unit’s low-pressure safety injection function. In this paper, metallurgical failure analysis technology was used, and crack origins were confirmed to be located at the root fillet by using a borescope. Scanning electron microscopy was used to observe the morphology of the tooth failure and determine it as fatigue fracture. At the same time, hardness testing and tensile strength testing were conducted, and it was found that the tensile strength of the tooth failure was insufficient. The microstructure of the gear section was observed by metallographic microscope and the gear teeth were measured, revealing that the root fillet of the sample did not meet the design standard requirements. By using MASTA software for design verification, it was determined that the small root fillet of the gear tooth caused stress concentration and was the main reason for frequent tooth failure. To solve the root cause, the equipment manufacturer optimized the mechanical properties of the material and increased the size of the root fillet, which completely solved the problem.