Purpose: Surgical vision–language foundation models learn generalizable representations from large-scale surgical data and support flexible adaptation to diverse downstream tasks by using text prompts as classifiers. However, their effectiveness is hindered by a critical semantic limitation: These models often struggle to distinguish between positive and negative textual assertions, a capability essential for fine-grained surgical tasks where the target object may occupy only a small and localized region. This limitation weakens the reliability of current state-of-the-art vision–language adaptation methods, leading to degraded performance when prompts require precise semantic discrimination. Methods: We propose Surg-NAT+, a few-shot vision–language adaptation framework designed to enhance foundation models for fine-grained surgical tasks. First, we introduce a negation-aware contrastive objective that explicitly strengthens the model’s ability to differentiate between affirmative and negated textual prompts. This objective is incorporated into pretrained models through multi-level adapter fusion (MAF), enabling hierarchical semantic refinement within the text encoder. Second, we propose a fine-grained self-distillation objective that improves visual grounding by enforcing consistency between global representations of image crops and the corresponding local patch embeddings. Results: We evaluate our approach on the Cholec80 dataset for few-shot surgical tool recognition using foundation models. Surg-NAT+ achieves state-of-the-art performance across all few-shot regimes, consistently surpassing existing baselines by a substantial margin. Qualitative analyses further demonstrate that our method yields more accurate visual grounding of target tools and enhances semantic separability in the learned feature space. Conclusions: We presented Surg-NAT+, a lightweight negation-aware adaptation framework that substantially improves the fine-grained recognition capabilities of surgical vision–language models in few-shot, multi-label tool recognition settings. Our approach provides an efficient and semantically robust pathway for adapting foundation models to surgical domains. The code is available at <a href="https://github.com/Yutongc-ai/fine-tune-SurgVLP">https://github.com/Yutongc-ai/fine-tune-SurgVLP</a>.

Surg-NAT+: negation-aware vision–language refinement for fine-grained surgical understanding

期刊论文
发表日期：

2026年6月14日

查看全文

Kun Yuan,
Yutong Cao,
Yunxi Tang,
Nicolas Padoy,
Xiangyu Chu

摘要

Purpose:

Surgical vision–language foundation models learn generalizable representations from large-scale surgical data and support flexible adaptation to diverse downstream tasks by using text prompts as classifiers. However, their effectiveness is hindered by a critical semantic limitation: These models often struggle to distinguish between positive and negative textual assertions, a capability essential for fine-grained surgical tasks where the target object may occupy only a small and localized region. This limitation weakens the reliability of current state-of-the-art vision–language adaptation methods, leading to degraded performance when prompts require precise semantic discrimination.

Methods:

We propose Surg-NAT+, a few-shot vision–language adaptation framework designed to enhance foundation models for fine-grained surgical tasks. First, we introduce a negation-aware contrastive objective that explicitly strengthens the model’s ability to differentiate between affirmative and negated textual prompts. This objective is incorporated into pretrained models through multi-level adapter fusion (MAF), enabling hierarchical semantic refinement within the text encoder. Second, we propose a fine-grained self-distillation objective that improves visual grounding by enforcing consistency between global representations of image crops and the corresponding local patch embeddings.

Results:

We evaluate our approach on the Cholec80 dataset for few-shot surgical tool recognition using foundation models. Surg-NAT+ achieves state-of-the-art performance across all few-shot regimes, consistently surpassing existing baselines by a substantial margin. Qualitative analyses further demonstrate that our method yields more accurate visual grounding of target tools and enhances semantic separability in the learned feature space.

Conclusions:

We presented Surg-NAT+, a lightweight negation-aware adaptation framework that substantially improves the fine-grained recognition capabilities of surgical vision–language models in few-shot, multi-label tool recognition settings. Our approach provides an efficient and semantically robust pathway for adapting foundation models to surgical domains. The code is available at https://github.com/Yutongc-ai/fine-tune-SurgVLP.