Purpose <p>This study investigates the bifurcation of an inerter-based damper with nonlinear damping and negative stiffness, examining its dynamic characteristics and vibration reduction performance under varying parameter conditions.</p> Methods <p>The slow-varying equations are derived using the complex variable averaging method, with investigations into the conditions for saddle-node bifurcation and Hopf bifurcation, as well as their impacts on the number and stability of equilibrium points. The periodic solution bifurcation of high-dimensional nonlinear systems is studied by combining the first integral method and Melnikov method. A parameter optimization strategy based on energy transfer is proposed through energy spectrum analysis, Poincaré maps, and time series plots.</p> Results <p>The study reveals the existence and evolutionary patterns of periodic orbits under parameter perturbations, as well as the bifurcation and stability of equilibrium points. By comparing with other vibration damping devices, it investigates the effects of key parameter variations on the energy transfer efficiency of the system.</p> Conclusions <p>The study demonstrates that the proposed nonlinear vibration model can significantly reduce the peak response, exhibiting excellent vibration reduction performance under harmonic excitation. It also provides valuable theoretical support for the stability, control strategies, parameter optimization, and periodic solution bifurcation mechanisms of vibration reduction systems in engineering applications.</p>