<p>Off-the-shelf visual representations have been widely applied in various tasks. However, as image retrieval involves a compact representation, the above practice does not obtain convincing performance, especially in realistic scenarios of novel domains and categories. In this paper, we make the first attempt to address it by organizing a generalized image retrieval task and proposing an off-the-shelf quantizer. Challenges of realizing them include two perspectives: visual inconsistency across domains and hidden semantics of unknown categories, which corrupt compact features. To tackle the former issue, we propose a cross-aligned contrastive learning objective for model training. It simultaneously reduces quantization error and domain gap, encouraging the model to generate domain-invariant quantization codes. To tackle the latter one, we design a <InlineEquation ID="IEq1"> <EquationSource Format="TEX">$2^{nd}$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msup> <mn>2</mn> <mrow> <mi mathvariant="italic">nd</mi> </mrow> </msup> </mrow> </math></EquationSource> </InlineEquation> order codebook which holds representative information of seen categories. Novel images are extracted by performing a compositional projection via all <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$2^{nd}$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msup> <mn>2</mn> <mrow> <mi mathvariant="italic">nd</mi> </mrow> </msup> </mrow> </math></EquationSource> </InlineEquation> order codewords, which improves generalization ability on unknowns. Combining them, we obtain a significant performance gain compared to the current state-of-the-art, where an up to <InlineEquation ID="IEq3"> <EquationSource Format="TEX">$8.0\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>8.0</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation> increase in mAP is observed. Code is available at: <a href="https://github.com/ZHlo-404/GIR-OTSQ">https://github.com/ZHlo-404/GIR-OTSQ</a>.</p>

Generalized Image Retrieval with Off-The-Shelf Quantizer

期刊论文
发表日期：

2026年6月16日

查看全文

Pengpeng Zeng,
Yihang Duan,
Xiaosu Zhu,
Jingkuan Song,
Lianli Gao,
Nicu Sebe,
Hengtao Shen

摘要

Off-the-shelf visual representations have been widely applied in various tasks. However, as image retrieval involves a compact representation, the above practice does not obtain convincing performance, especially in realistic scenarios of novel domains and categories. In this paper, we make the first attempt to address it by organizing a generalized image retrieval task and proposing an off-the-shelf quantizer. Challenges of realizing them include two perspectives: visual inconsistency across domains and hidden semantics of unknown categories, which corrupt compact features. To tackle the former issue, we propose a cross-aligned contrastive learning objective for model training. It simultaneously reduces quantization error and domain gap, encouraging the model to generate domain-invariant quantization codes. To tackle the latter one, we design a $2^{nd}$ $2^{nd}$ order codebook which holds representative information of seen categories. Novel images are extracted by performing a compositional projection via all $2^{nd}$ $2^{nd}$ order codewords, which improves generalization ability on unknowns. Combining them, we obtain a significant performance gain compared to the current state-of-the-art, where an up to $8.0\%$ $8.0 %$ increase in mAP is observed. Code is available at: https://github.com/ZHlo-404/GIR-OTSQ.