<p>Viseme is a visual unit of speech that describes the movement of the lip when uttering words or sentences. Accurate viseme classification and word recognition are mandatory for proper speech understanding, particularly in applications such as aiding communication for the hearing impaired and enhancing human-computer interaction. Prior, various deep learning-based methodologies are developed for viseme classification and word detection. However, they often struggle with low accuracy in word/sentence detection, fail to predict the correct sentences/words grammatically and contextually, and require higher computational time. To overcome this issue, a novel Carnegie Mellon Pronouncing dictionary-based VisemeWNet strategy is proposed for classifying the visemes and recognizing the word. The proposed VisemeWNet model utilized a Gated Dilated Capsule Attention Network for the classification of visemes that integrates a Gated Dilated Convolution for capturing the high-density features and multi-scale information, Transition layer for enhancing computational efficiency, and an Adaptive Capsule Attention Network for concentrating the spatial and channel attention’s pertinent features in order to improve the ability of the model to prioritize relevant information within the input data. Additionally, the proposed VisemeWNet framework utilized a Contextformer to detect the words/sentences that are grammatically and contextually correct and it incorporates a Generative Pre-trained Transformer with relative attention for calculating the perplexity score, and Bidirectional Encoder Representation from Transformer for measuring the semantic coherence. The proposed VisemeWNet method is evaluated on the MIRACL-VC1 dataset. The result demonstrates that the proposed VisemeWNet framework effectively classified the visemes and efficiently detected the words/sentences grammatically and contextually. Moreover, the proposed model achieved a higher Word Accuracy Rate of 98.8%, a higher Sentence Accuracy Rate of 92.5%, and less Viseme Error Rate of 3.5%. These results highlight the potential of the VisemeWNet framework to improve real-time communication systems for the hearing impaired and enable more interactions in human-computer systems.</p>