Hand gestures are widely used across various applications, and research on hand gesture recognition continues to evolve due to its usability. However, challenges such as occlusion remain significant. Therefore, this study aims to introduce a novel approach using pose estimation to evaluate distraction levels based on rate, duration, and saliency while addressing occlusion-related issues to enhance keypoint detection accuracy for more reliable hand gesture analysis. To achieve this, a custom dataset was built, and data augmentation techniques were applied to enhance the robustness of the “Palms Open” pose analysis. Additionally, a Kalman filter was implemented to track missing keypoints, while a Generative Convolutional Network (GCN) was utilized to infer them, improving overall accuracy. The results demonstrated that the model achieved a mAP@50 of 97.70% and 97.79% accuracy. The integration of the GCN excelled, achieving 32 mm for Mean Per Joint Position Error (MPJPE), 94.78% for Object Keypoint Similarity (OKS), and 96.80% for the Percentage of Correct Keypoints (PCK). As a practical application, the model was implemented in a Flask environment, incorporating mathematical computations to assess distraction levels using five sample videos. The findings indicated that hand gesture dynamics ranged from “moderate” to “very high” distraction. In conclusion, this solution can assist experts in evaluating the impact of hand movements and serve as a benchmark for further improvements.