EMFIN: A Multimodal Fusion Algorithm and Intelligent Navigation System for Intelligent Chemistry Experiments
摘要
There are many safety issues in current experimental teaching that need to be urgently addressed. To tackle these problems, our team proposes a virtual-reality fusion intelligent experiment platform. However, existing intelligent experiment platforms still face limitations, such as restricted student operation methods and weak system-level intent understanding. Therefore, this thesis introduces the application of an interactive smart beaker device in virtual teaching, along with a multimodal intent understanding and human-computer interaction algorithm based on the smart beaker. The main innovation of this thesis lies in the design and development of a virtual-reality fusion intelligent experiment platform based on the smart beaker. This platform can sense user behaviors and map them into the virtual interface. A multimodal fusion algorithm is also proposed. It first extracts multi-source information from the sensor and speech channels, and then uses an improved Dempster-Shafer (DS) evidence theory model to fuse the data, aiming to accurately recognize user intent. Finally, based on the operational procedures of middle and high school chemistry experiments, an intelligent navigation system is designed to monitor and correct user errors in real time. According to the statistical results of user intent recognition accuracy, the proposed model achieves an average recognition accuracy of 95.74%, providing a new intelligent teaching model for chemistry education in middle and high schools.