<p>Federated semantic segmentation (FSS) has emerged as a promising solution for collaborative model training across distributed devices while preserving data privacy. However, current FSS frameworks fundamentally require homogeneous semantic segmentation labels across all clients, posing key challenges for cross-institutional collaboration with heterogeneous type label(semantic/instance/panoptic segmentation) across organizations. To bridge this critical gap, we propose a novel FSS framework that enables collaborative learning across multiple label types within a unified architecture, named as FedMTL. To address noise in pseudo-labeling, we propose a confident pseudo-label generation algorithm that rectifies region-level semantics through dominant category distribution analysis, achieving instance-guided adaptive label refinement. The proposed adaptive weight aggregation strategy resolves client contribution imbalance by foreground-density-aware weighting, where local models with elevated semantic saliency receive prioritized coefficients to enhance discriminative feature propagation. Experimental validation on Pascal VOC and Cityscapes demonstrates FedMTL’s superior performance, with maximum improvements of +3.1% mIoU on Pascal VOC and +2.7% mIoU on Cityscapes over FedAvg.</p>