Magnetic resonance imaging (MRI) can detect subtle structural brain changes associated with neurodegenerative disorders up to a decade before clinical symptoms appear, making it a vital tool for early intervention. Longitudinal segmentation enables precise quantification of these changes; however, the task is challenging due to the faint nature of brain atrophy (0.5–1.0% per year) and the potential for inter-scan noise to obscure clinically meaningful patterns. In this work, we introduce two novel temporal consistency loss functions for longitudinal brain segmentation: a volume-based loss (VL) that promotes consistent volume trajectories without population-level normalization and a signed distance field (SDF) loss that enforces expected structural shrinkage without relying on age or time intervals. Unlike prior methods, our framework handles variable gaps between timepoints, making it more applicable to real-world clinical data. Our method is evaluated using group separation (Cohen’s d), both cross-sectionally and longitudinally, which more accurately reflect longitudinal trends and provides a more clinically meaningful metric compared to traditional Dice scores. This provides a robust and adaptable framework for clinically meaningful longitudinal brain segmentation.