The rapid cycling synchrotron (RCS) at the China spallation neutron source operates as a high-intensity proton accelerator. The coupled bunch instability was observed during RCS beam commissioning, which significantly limited the beam power. To investigate the dynamics of instability under an increased beam power, a pulsed octupole magnet with a gradient of 900 \({\text {T/m}^{3}}\) was developed. The magnet system integrated an octupole magnet with a pulsed power supply. The field was carefully measured to examine the performance before its installation into the tunnel. After the installation of the magnets, beam measurements were performed to confirm the effectiveness of the instability mitigation on an actual proton beam. The measurement results show that the instability can be suppressed using the pulsed octupole magnet, particularly at the high-energy stage in an acceleration cycle, meeting the requirements for stable operation of the accelerator. Additionally, when the instability is completely suppressed through chromaticity optimization, octupole magnets can significantly enhance the RCS transmission efficiency, which is crucial for controlling beam loss. The pulsed octupole magnet offers significant progress in beam stability in the RCS, providing valuable experience for further beam power enhancement.
