In the field of Active Assisted Living (AAL), maintaining the privacy of individuals while effectively monitoring their health and mobility status remains an unsolved question. In medicine and gerontology, several metrics have been developed to assess mobility of older adults, and the relationship between these metrics and future outcomes has been established with some success. This chapter delves into the utilization of video-based methods for gait and frailty recognition, presenting a comprehensive overview of techniques, challenges, and advancements in the field. By analysing patterns in gait, subtle changes indicative of frailty or potential health deterioration can be detected early, facilitating timely interventions and personalized care. However, the deployment of video-based gait recognition systems in AAL environments requires a delicate balance between the efficacy of monitoring and safeguarding individual privacy. This chapter analyses existing frailty metrics and their potential use in automated ecological momentary assessment from heterogeneous sensors, including video, and with a high emphasis on privacy preservation when doing so.