Collapse of the femoral head is a critical event in osteonecrosis (ONFH) that often leads to debilitating hip pain and necessitates total hip arthroplasty. Early and accurate prediction of collapse risk is crucial for personalized treatment planning. While many studies focus on the automated diagnosis of ONFH, prognosis remains less explored. In this study, we propose a robust tri-stream deep learning framework that extracts features from T1-weighted MRI, region-of-interest (ROI) labels, and ONFH grades to estimate patient-specific collapse risk. We introduce an independent Spatial Label Encoder (SLE) module that tokenizes discrete ROI labels into dense, context-rich embeddings, thereby facilitating multi-modality model training. Experiments on 92 hips (70 patients) show that our approach performs competitively with state-of-the-art (SOTA) methods across most metrics, achieving a concordance index (CI) of 0.847±0.087 and an integrated AUC of 0.884 in 5-fold cross-validation. Notably, the SLE module enhances long-term discrimination by up to \(2.4\%\) on AUC at 60 months compared to our base network. These findings highlight the potential benefits of late-fusion strategies with label tokenization for predicting femoral head collapse in ONFH, contributing to improved early intervention and prognosis.