Visual monitoring in Ambient Assisted Living (AAL) environments demands context-aware systems capable of adapting to real-world variability. Traditional models relying on fixed sensors or unimodal reasoning often fall short in dynamic settings. To this end, we propose a novel transformer-based multi-modal fusion framework, ContextualVQA, for intelligent monitoring via Visual Question Answering (VQA). It integrates ViLT’s patch-based vision encoding, LXMERT’s object-level reasoning, and Masked Language Modeling (MLM) for refined textual understanding. By leveraging natural language interfaces and visual grounding, the system enables intuitive interaction without requiring technical expertise. Experiments on NTU RGB+D and SVideoQA demonstrate its robustness and superior performance over existing baselines in complex monitoring scenarios.