A major challenge of wearable human activity recognition (WHAR) lies in the fact that many existing personalization techniques rely heavily on labeled activity data or physical details of the target subject, which necessitates additional efforts at deployment time. This is often infeasible in practical applications, either due to the lack of input modalities in wearables or due to additional efforts for the users in personalizing their device. To address this problem, we highlight the use of Unsupervised Personalized Deep Learning techniques to enhance the performance of existing Deep Learning WHAR models after initial deployment. This architecture does additional user feedback and can be exploited to improve model performance using only collected sensor data. Our approach identifies samples that present predictive challenges to the original model, trains a surrogate model utilizing data from the training set that is similar to the identified samples, and corrects the predictions of the identified samples through the surrogate model. To validate the effectiveness of the proposed approach, we compare our results against one state-of-the-art Unsupervised Domain Adaptation approach and two state-of-the-art unsupervised personalization approaches using six human activity recognition data sets as benchmarks. Experimental results demonstrate that the proposed method outperforms the state-of-the-art approaches in the unsupervised subject adaptation task. Furthermore, the proposed approach not only attains optimal performance across the majority of data sets but also manifests significant advantages, particularly in terms of stability, across various metrics and data sets.