Under the actual complex operating conditions, insulation defects tend to appear at the edge of the RIP bushing pole plate, leading to partial discharge and thus causing insulation failure. It has been a critical issue to master the epoxy resin insulation characteristics of the valve-side RIP bushing under composite stress, which determines its safe and stable operation. To further investigate the internal epoxy resin insulation characteristics, an experimental platform for electrical treeing observation and partial discharge measuring has been established. A needle plate model is constructed, and an epoxy resin sample is prepared to simulate the internal defects of the valve-side RIP bushing. The composite voltage with multi-harmonic components is used to conduct epoxy resin electrical tree growth insulation degradation experiments, and the experimental results are compared with those under AC voltage. The experimental results show that with an increase in experimental temperature and the addition of higher harmonics, the electrical tree initiation voltage and breakdown time of epoxy resin are significantly reduced. The morphology of electrical trees changes from branch to clump-branch, with the fractal dimension and expansion coefficient gradually increasing. In the same fundamental frequency cycle, the phase distribution of electrical tree partial discharge measured under composite voltage exhibits “multi-peak” characteristics compared to that of AC voltage. By comparing experimental results under different voltage waveforms, it is found that the composite voltage containing multiple harmonics has a more severe impact on epoxy resin materials than AC voltage.