Pre trained language models (PLMs) inherit societal regularities from web scale corpora and often encode group dependent associations. A persistent practical question is: Why do those biases frequently survive standard fine tuning? We provide a diagnostic explanation consistent with optimization moving the model along a low dimensional, task aligned set of directions, while bias encoding directions receive little update energy. We formalize this behavior as Minimal Invariant Subspaces (MIS): protected attribute information occupies subspaces in representation space that remain largely invariant under standard fine tuning. We instantiate MIS with two simple measurements: (i) a gradient projection ratio that quantifies how much of the training signal falls along a pre identified bias direction; and (ii) subspace stability via principal angles / Grassmann distance between pre and post fine tuning bias subspaces. Our theoretical proposition shows that, in linearized first order dynamics, if the task gradient is orthogonal (up to \(\epsilon \) ) to a bias subspace at each step, first order updates preserve that subspace. We conclude that, absent explicit intervention, standard fine tuning need not be expected to erase bias encoding directions.