Catastrophic forgetting is a persistent obstacle for continual learning on memory-constrained edge devices. We introduce an extension of Lightweight Singular Value Decomposition Generators (SVD Generators), named Multi-Subspace SVD Generators (MSSG), that replaces each single global SVD with a small ensemble of low-rank SVDs per task and class. Each subspace captures a local linear patch; together they approximate the non-linear data manifold while keeping storage cost proportional to the rank, not the number of replay samples. A closed-form memory model allows MSSG to trade subspace count and rank against an equivalent raw-sample buffer, enabling fair comparison to experience-replay baselines. Across five image benchmarks; MNIST, Fashion MNIST, NOT MNIST, CIFAR10 and Tiny ImageNet, MSSG (i) outperforms its single-generator predecessor, (ii) matches or exceeds the accuracy of Experience Replay with a large buffer, and (iii) does so with less than one-tenth of their memory footprint. Because MSSG stores only compact factorised statistics, both rehearsal and generator updates run in milliseconds on resource-limited hardware, making it a practical drop-in replacement for replay buffers in on-device lifelong learning applications.