Scholarlink

<p>In recent years, vision-language models (VLMs) have been applied to various fields, including healthcare, education, finance, and manufacturing, with remarkable performance. However, concerns remain regarding VLMs’ consistency and uncertainty, particularly in critical applications such as healthcare, which demand a high level of trust and reliability. This paper proposes a novel approach to evaluate uncertainty in VLMs’ responses using a convex hull approach on a healthcare application for visual question answering (VQA). For any VLM, temperature refers to a sampling parameter used in probabilistic generation, which controls the randomness of the model’s output. The LLM-CXR model is selected as the medical VLM utilized to generate responses for a given prompt at different temperature settings. According to the results, the LLM-CXR VLM shows high uncertainty at higher temperature settings, which can be characterized geometrically in feature space. Experimental results emphasize the importance of uncertainty in VLMs’ responses, especially in healthcare applications.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysis

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月29日

查看全文
  • Ferhat Ozgur Catak,
  • Murat Kuzlu,
  • Taylor Patrick,
  • Michel Audette

摘要

In recent years, vision-language models (VLMs) have been applied to various fields, including healthcare, education, finance, and manufacturing, with remarkable performance. However, concerns remain regarding VLMs’ consistency and uncertainty, particularly in critical applications such as healthcare, which demand a high level of trust and reliability. This paper proposes a novel approach to evaluate uncertainty in VLMs’ responses using a convex hull approach on a healthcare application for visual question answering (VQA). For any VLM, temperature refers to a sampling parameter used in probabilistic generation, which controls the randomness of the model’s output. The LLM-CXR model is selected as the medical VLM utilized to generate responses for a given prompt at different temperature settings. According to the results, the LLM-CXR VLM shows high uncertainty at higher temperature settings, which can be characterized geometrically in feature space. Experimental results emphasize the importance of uncertainty in VLMs’ responses, especially in healthcare applications.