Sign Language Recognition (SLR) plays a crucial role in improving accessibility for individuals with hearing impairments, particularly within Smart Home environments. This paper presents Smart-LSE-UAL, a novel real-time Spanish Sign Language (LSE) recognition system designed to facilitate interaction with smart home devices. The proposed system leverages a hybrid CNN-LSTM neural network architecture to process real-time videos. The keypoints required for recognition (hands and body) are extracted using MediaPipe, ensuring a compact and privacy-preserving representation of user gestures. A newly recorded LSE dataset, specifically tailored for smart home control, is introduced to enhance recognition accuracy. The system is fully integrated with a KNX-based smart home infrastructure at the University of Almería, enabling seamless control of household components, such as lights, shutters, and air conditioning, via sign language commands. Experimental evaluations demonstrate the effectiveness of the proposed approach, achieving high accuracy in gesture recognition for different users, while maintaining real-time performance.