<p>Human–computer interaction (HCI) is a cutting-edge and useful research frontier. This study introduces a novel HCI framework called Kin-LeapK. The framework aims to address three critical limitations of conventional single-sensor systems. These limitations are performance degradation under adverse conditions, insufficient spatial feature extraction, and low robustness in multi-sensor interaction signal processing. As for the interaction information recognition methods, our key contributions based on improved AdaBoost methods are: (1) hand gesture-based interaction: a dimension-by-dimension reverse processing (DDRP) strategy refines the Cuckoo Search Algorithm (CSA) for Support Vector Machine (SVM) hyperparameter optimization, establishing the CSA-SVM-AdaBoost framework to address multi-scale gesture variability. (2) Action-based interaction: incorporating a Gaussian mutation factor into Lévy flight-distributed SVM kernels produces the Lévy-SVM-AdaBoost classifier. It enhances robustness to occlusions and posture ambiguities. (3) Speech-based interaction: a modified Sparrow Search Algorithm (SSA) optimizes Bi-directional Long Short-Term Memory (BiLSTM) temporal dependencies, forming the SSA-BiLSTM-AdaBoost ensemble to improve noisy speech signal discrimination significantly. Experimental validation demonstrates superior performance: the framework achieves a 98.80% recognition accuracy for eight core hand gestures, 95.91% recognition accuracy for four basic interaction actions, and 96.22% recognition accuracy for speech signals. These results demonstrate their state-of-the-art performances and great potential.</p>