<p>Zero-shot segmentation has shown strong performance by generating features from semantic embeddings to adapt models to unseen classes. These generated features are typically aligned with the visual distribution of seen classes to improve generalization on visual features. However, this vision-centric alignment may overfit to seen classes due to the absence of visual data for unseen classes. To address this, we propose a semantic-centric alignment method that aligns generated features with a well-structured semantic distribution spanning all classes. First, we align vision backbone features with CLIP tokens via Vision-to-CLIP alignment. This alignment leverages CLIP’s vision-language matching capabilities to produce semantically aligned backbone features. Next, we generate synthetic features from semantic embeddings for unseen classes. These features are supervised by semantically aligned visual features and CLIP semantic embeddings to enhance visual diversity while preserving semantic consistency. Finally, we finetune the class projector using real features from seen classes and pseudo features from unseen classes to improve generalization and reduce overfitting. Our semantic-centric alignment improves zero-shot generalization by constructing a unified and semantically aligned feature space. With limited data, our method achieves state-of-the-art performance in zero-shot panoptic segmentation and readily extends to zero-shot semantic segmentation and achieves great generalization capability across datasets.</p>