HAR plays a crucial role in numerous applications, especially in uncontrolled settings where conventional deep learning models tend to falter due to high resource requirements and overfitting issues. This chapter presents a streamlined and efficient deep learning model that utilizes a hybrid feature fusion strategy to overcome these difficulties. By combining handcrafted features with deep learning features, the model successfully identifies complex activity patterns while keeping computational demands low. We developed a unique dataset leveraging smartphone sensors in real-world contexts, in addition to using public datasets for thorough evaluation. Our suggested hybrid deep learning model proficiently extracts both spatial and temporal characteristics from unprocessed sensor data, achieving an impressive 98% accuracy while significantly reducing computation time in comparison with standard models like DNN and RNN. This research highlights the effectiveness of our method in improving HAR performance in real-world situations, underlining its suitability for implementation on devices with limited resources.