Blind omnidirectional image quality assessment (BOIQA) has been a challenging problem in the image quality assessment field, due to the geometric characteristic of omnidirectional images (OIs) and complicated human behavior in immersive experience. Toward solving this problem, we resort to Multimodal Large Language Models (MLLMs), which show great success in both computer vision and natural language processing, while they have not been investigated in BOIQA. Specifically, we first generate coarse and detailed quality-aware descriptions for OIs by feat of MLLMs to get richer information, instead of simple quantitative scalars. Upon the generated text descriptions and the paired images, we fine-tune a top-performing model (i.e., Long-CLIP) under the general contrastive learning framework, mining robust and representative embeddings in the vision-language space. Then, we design a family of MultiModal BOIQA (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">$\textrm{MMBO}$</EquationSource> <EquationSource Format="MATHML"><math> <mtext>MMBO</mtext> </math></EquationSource> </InlineEquation>) models based on the embeddings in the built vision-language space, comprehensively investigating the effectiveness of text features, visual features, and their interaction in capturing quality degradation of OIs. Experimental results on two large-scale OIQA databases demonstrate the superior performance of <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$\textrm{MMBO}$</EquationSource> <EquationSource Format="MATHML"><math> <mtext>MMBO</mtext> </math></EquationSource> </InlineEquation> models, e.g., the best performing <InlineEquation ID="IEq3"> <EquationSource Format="TEX">$\textrm{MMBO}$</EquationSource> <EquationSource Format="MATHML"><math> <mtext>MMBO</mtext> </math></EquationSource> </InlineEquation> model outperforms the second-ranked method 9.4<InlineEquation ID="IEq4"> <EquationSource Format="TEX">$\%$</EquationSource> <EquationSource Format="MATHML"><math> <mo>%</mo> </math></EquationSource> </InlineEquation> and 6.6<InlineEquation ID="IEq5"> <EquationSource Format="TEX">$\%$</EquationSource> <EquationSource Format="MATHML"><math> <mo>%</mo> </math></EquationSource> </InlineEquation> on these two OIQA databases in terms of PLCC, respectively, and shows promising generalizability in cross-database validation and gMAD competition.

Blind Omnidirectional Image Quality Assessment: Embracing the Magic Power of Multimodal Large Language Models

期刊论文
发表日期：

2026年1月21日

查看全文

Jiebin Yan,
Jiayu Zhang,
Junjie Chen,
Pengfei Chen,
Xuelin Liu,
Ziwen Tan,
Yuming Fang

摘要

Blind omnidirectional image quality assessment (BOIQA) has been a challenging problem in the image quality assessment field, due to the geometric characteristic of omnidirectional images (OIs) and complicated human behavior in immersive experience. Toward solving this problem, we resort to Multimodal Large Language Models (MLLMs), which show great success in both computer vision and natural language processing, while they have not been investigated in BOIQA. Specifically, we first generate coarse and detailed quality-aware descriptions for OIs by feat of MLLMs to get richer information, instead of simple quantitative scalars. Upon the generated text descriptions and the paired images, we fine-tune a top-performing model (i.e., Long-CLIP) under the general contrastive learning framework, mining robust and representative embeddings in the vision-language space. Then, we design a family of MultiModal BOIQA ( $\textrm{MMBO}$ $MMBO$ ) models based on the embeddings in the built vision-language space, comprehensively investigating the effectiveness of text features, visual features, and their interaction in capturing quality degradation of OIs. Experimental results on two large-scale OIQA databases demonstrate the superior performance of $\textrm{MMBO}$ $MMBO$ models, e.g., the best performing $\textrm{MMBO}$ $MMBO$ model outperforms the second-ranked method 9.4 $\%$ $%$ and 6.6 $\%$ $%$ on these two OIQA databases in terms of PLCC, respectively, and shows promising generalizability in cross-database validation and gMAD competition.