In this study, we develop Robo-Sax system, which is a semi-automatic instrument-playing robot that performs key fingerings automatically while the human player handles the blowing motion. We also create four versions of Robo-Sax (soprano, alto, tenor, and baritone) and propose a system for ensemble playing using a saxophone quartet. Our goal is to design the hardware and develop the system software to ensure practical applicability. Experiments with beginner, intermediate, and professional saxophonists are conducted to demonstrate the system’s potential.