Elderly and people with visual deficiencies often encounter challenges with traditional digital interfaces, limiting their ability to effectively engage with technology. A multimodal approach integrating advanced gesture recognition with voice command processing offers a more intuitive and user-friendly alternative, leveraging deep learning models to enhance accuracy and responsiveness. The designed system provides highly accurate gesture and voice command recognition, correctly identifying gestures 95.6% of the time and processing spoken commands with 97.2% accuracy. It maintains a Word Error Rate (WER) of 3.8%, ensuring precise speech recognition. The system processes gestures in under 40 ms and voice commands in approximately 85 ms, ensuring smooth and efficient interaction. Even in noisy environments with background noise levels up to 30 dB, it maintains an impressive 92.5% accuracy, demonstrating its reliability in real-world applications. By analysing user-specific patterns, the model continuously improves recognition accuracy, reducing gesture misclassification rates to 7.5% and adapting to different speech styles and accents. The combination of computer vision, automatic speech recognition, and deep learning methods enables user interaction for users with different motor abilities and speech clarity. The system can also accommodate multiple languages and recognize a wide range of hand gestures, thus enhancing inclusivity for users of different cultural and linguistic backgrounds. With greater autonomy, safety, and enjoyment, the system simplifies interactions with smart devices, making them simpler and more intuitive to use. These advancements open the door to future innovations in assistive technologies, smart home automation, and healthcare applications, thus propelling the field of human-computer interaction.