<p>Semantic segmentation serves as a foundational technology for autonomous vehicles by providing pixel-level scene parsing capabilities, which are crucial for the precise perception and understanding of complex road environments. Modern semantic segmentation models generally achieve good results, but variables such as fluctuating lighting conditions and unexpected anomalies in the real world can lead to segmentation errors with serious consequences. To address this, we investigated intrinsic failures and anomalies in semantic segmentation and proposed a unified framework that manages these errors specifically for autonomous vehicles. By sorting the output masks into inlier and void categories, we used post-processing error scores from the inlier masks to accurately detect failures. Additionally, we developed an anomaly fusion module that integrates data from both inlier and void masks, alongside pyramid features from the backbone, ultimately delivering impressive anomaly detection capabilities. Extensive experiments prove that our approach outperforms current failure detection methods and achieves best results among techniques that do not require retraining the original segmentation models.</p>