This article explores the application of computer vision technology in user interface design, particularly through gesture recognition to enhance human–computer interaction. Gestures offer natural and versatile advantages in the process of human–computer interaction. Vision-based gesture recognition technology has broad application potential in virtual environments, sign language recognition, machine control, and other fields. In the task of gesture recognition in user interface design, the recognition results are affected by some irrelevant factors, such as dynamic backgrounds and close-fitting clothing. The attention mechanism is a good way to address the above problems, as it applies weights to key areas to extract essential features, thereby improving the model's performance. Several commonly used attention mechanisms, such as spatial attention, channel attention, and mixed attention mechanisms, can only apply attention to a single-frame image, focusing on specific regions. However, gestures consist of sequences of multiple-frame images, and the recognition results are collectively influenced by multiple-frame gesture images. The aforementioned attention mechanisms cannot meet the requirements effectively. To address this issue, a joint intra-frame and inter-frame attention module has been investigated. The model is able to prevent the impact of irrelevant elements during the gesture identification process because of this module's ability to apply attention concurrently across many consecutive frames and inside a single frame. In order to further user interface design, this study employs convolutional neural networks to include gesture recognition. These results shed light on how computer vision technology may revolutionize UI design and enhance user experiences.