Scholarlink

Blind image quality assessment (BIQA) has recently been significantly improved by leveraging vision-language models (VLMs). Given their semantic reasoning capabilities, it is tempting to assume that these models extract visual features, generate descriptive texts, and perform quality inference in a human-like manner. However, such models sometimes produce textual descriptions of visual attributes that contradict the final quality predictions (e.g., the model outputs “the image is clear” but finally assesses the quality as “poor”). Moreover, the predicted quality often changes unstably during the inference process. These behaviors are obviously inconsistent with human reasoning. To gain insights into the gap between the models and humans, this study investigates what causes contradictory quality assessments and why the assessments are susceptible to change. We first estimate the relationship between the final quality assessments and the generated visual features. Our analysis reveals that the final predictions are, to some extent, not derived through reasoning based on these features, so that the logical correlation between them is relatively weak. Furthermore, by decoding the intermediate layers of the VLM, we observe that the language model often relies only on a limited set of candidate tokens. This behavior further explains why the quality assessments are susceptible to change. To promote human-like logical reasoning in VLMs, we next introduce a two-stage tuning method that explicitly decouples visual perception from quality inference. The VLM is instructed to learn the visual features and conclude the quality solely based on the visual features, respectively, in the two stages. Experimental results on the standard IQA datasets (SPAQ and KONIQ) show that our approach reduces prediction instability from 22.00% to 12.39%, and achieves average improvements of 0.3124/0.3507 in terms of SRCC/PLCC across four standard datasets (LIVE, CSIQ, SPAQ, and KONIQ), compared to the baseline. Further analysis and visualization provide evidence that our proposed method not only enhances the model’s stability but also builds a reliable inference.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Building Reasonable Inference for Vision-Language Models in Blind Image Quality Assessment

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Yuan Li,
  • Zitang Sun,
  • Yen-ju Chen,
  • Shin’ya Nishida

摘要

Blind image quality assessment (BIQA) has recently been significantly improved by leveraging vision-language models (VLMs). Given their semantic reasoning capabilities, it is tempting to assume that these models extract visual features, generate descriptive texts, and perform quality inference in a human-like manner. However, such models sometimes produce textual descriptions of visual attributes that contradict the final quality predictions (e.g., the model outputs “the image is clear” but finally assesses the quality as “poor”). Moreover, the predicted quality often changes unstably during the inference process. These behaviors are obviously inconsistent with human reasoning. To gain insights into the gap between the models and humans, this study investigates what causes contradictory quality assessments and why the assessments are susceptible to change. We first estimate the relationship between the final quality assessments and the generated visual features. Our analysis reveals that the final predictions are, to some extent, not derived through reasoning based on these features, so that the logical correlation between them is relatively weak. Furthermore, by decoding the intermediate layers of the VLM, we observe that the language model often relies only on a limited set of candidate tokens. This behavior further explains why the quality assessments are susceptible to change. To promote human-like logical reasoning in VLMs, we next introduce a two-stage tuning method that explicitly decouples visual perception from quality inference. The VLM is instructed to learn the visual features and conclude the quality solely based on the visual features, respectively, in the two stages. Experimental results on the standard IQA datasets (SPAQ and KONIQ) show that our approach reduces prediction instability from 22.00% to 12.39%, and achieves average improvements of 0.3124/0.3507 in terms of SRCC/PLCC across four standard datasets (LIVE, CSIQ, SPAQ, and KONIQ), compared to the baseline. Further analysis and visualization provide evidence that our proposed method not only enhances the model’s stability but also builds a reliable inference.