Recent advances in pathology foundation models have demonstrated remarkable capabilities in feature representation through large-scale pre-training. Despite their success, these models often suffer from substantial computational overhead, posing challenges for deployment in practical clinical settings with resource limitations. In addition, their generalizability is limited by distributional biases inherent in the pretraining datasets, often resulting in suboptimal performance when transferred to rare or underrepresented cancer types. To overcome these limitations, we introduce Universal Distillation (UniDis), a novel framework that distills knowledge from multiple large foundation models into a lightweight model with significantly fewer parameters. UniDis supports efficient and personalized fine-tuning on private, institution-specific datasets, enabling tailored adaptation to various downstream cancer-type classification tasks. Extensive experiments on the TCGA-Lung and CPTAC-Lung datasets demonstrate that UniDis achieves state-of-the-art performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

UniDis: Universal Distillation for Efficient and Personalized Pathology Diagnosis

  • Rongze Ma,
  • Mengkang Lu,
  • Yong Xia

摘要

Recent advances in pathology foundation models have demonstrated remarkable capabilities in feature representation through large-scale pre-training. Despite their success, these models often suffer from substantial computational overhead, posing challenges for deployment in practical clinical settings with resource limitations. In addition, their generalizability is limited by distributional biases inherent in the pretraining datasets, often resulting in suboptimal performance when transferred to rare or underrepresented cancer types. To overcome these limitations, we introduce Universal Distillation (UniDis), a novel framework that distills knowledge from multiple large foundation models into a lightweight model with significantly fewer parameters. UniDis supports efficient and personalized fine-tuning on private, institution-specific datasets, enabling tailored adaptation to various downstream cancer-type classification tasks. Extensive experiments on the TCGA-Lung and CPTAC-Lung datasets demonstrate that UniDis achieves state-of-the-art performance.