The necessity for a broader acceptance of technology has been focused on by the rapid expansion of human–computer interaction (HCI), particularly for those with motor disabilities. These users find it difficult to interact with digital environments while using conventional input devices like keyboards and mouse. This work uses cutting-edge technologies including MediaPipe, OpenCV, computer vision, and deep learning to demonstrate an innovative approach for computer control via hand gestures and sign language understanding. Without the need for additional hardware, the system translates hand movements in real time into commands, allowing for virtual keyboard input, virtual drawing, cursor control, and game control. This research has contributed much to the current work by incorporating Indian Sign Language (ISL) translation in real time. It is meant more significantly to support a deaf and hard-of-hearing population in communicating more effectively. An ISL gesture recognition system with its translating capabilities into text and voice communication, in various Indian languages, might make access more effective. Furthermore, the system’s functionality is improved by an AI-integrated speech bot that processes voice instructions and offers interactive feedback. With a solution that is adaptable and user-friendly, the project seeks to close the gap between human capacities and computer interfaces, serving a wide range of user demographics. This system has the potential to have a substantial impact on gaming, creative arts, accessibility technology, and general HCI by improving the accessibility and intuitiveness of digital interaction. This could lead to a more inclusive digital society. The necessity for more inclusive and accessible technology has been highlighted by the rapid expansion of human–computer interaction (HCI), particularly for those with motor disabilities. These users find it difficult to interact with digital environments while using traditional input devices like keyboards and mice. This work uses cutting-edge technologies including MediaPipe, computer vision, and deep learning to present a revolutionary method for computer control via hand gestures and sign language. This research presents a novel gesture-based human–computer interaction system that combines hand gestures, eye gestures, and sign language to bridge the gap between physical limitations and digital accessibility. Leverage advanced technologies such as MediaPipe, OpenCV, and deep learning in the system to provide an inclusive platform for diverse-user demographics, including those with motor impairments. The system achieved an accuracy of 94.8% in the detection of gestures with variations of different kinds. That is, it had a comparative accuracy in the process of translation by ISL to be about 91.2%. This is highly impressive under scenarios that are average; still, minimal loss in the effectiveness under full darkness and crowded settings means that there’s scope for improvement in it. This makes the input modalities highly usable in gaming, creative arts, and assistive applications and pushes forward HCI with a more natural and intuitive interface. It presents future work in gesture-based systems that will bring digital inclusivity and redefine accessibility standards.