In special education, the social skills and learning needs of deaf and mute students have always been a focus of attention. With the advancement of educational reforms, many learning tasks and communication activities are conducted through conference systems. However, traditional 2D conference systems are not friendly to special students, as their physiological challenges often lead to feelings of inferiority in communication, affecting participation effectiveness. To address this issue, we propose a Three-Dimensional Interactive Virtual Conference System (3DIVCS), which not only provides video, voice communication, and speech recognition features, but also enables users to interact and engage on stage as 3D virtual characters. To validate the effectiveness of 3DIVCS for deaf and mute individuals, researchers developed a set of 3DIVCS applications and conducted real-world experiences, surveys, interviews, and analyses using independent t-tests with 20 deaf and mute students. The results indicate that 3DIVCS significantly improved their learning outcomes in conferences, enhanced their confidence, and ultimately elevated their sense of participation and communication quality.