This work presents a hand gesture recognition system using relative hand marks to classify static and dynamic gestures from both webcam streams and upload the static images. The hand gestures collected in the existing works may have low lighting conditions, various environments and static gestures. The proposed real-time gesture recognition system incorporates relative hand landmarks to enhance the system’s tracking accuracy. The system uses MediaPipe's hand landmark detection to extract key points on the hand, which are then classified using pre-trained models. The system also implements the use of point history to recognize motion gestures. The system normalizes collected hand landmarks to ensure reliability against variations in hand size, location, orientation and poor lighting. The experimental results demonstrate our system's effectiveness in recognizing a vocabulary of gestures, showcasing its potential for real-time applications in the fields of hand gesture recognition and human–computer interaction.