Open-Vocabulary Object Detection (OVOD) aims to detect objects of unseen classes through pre-trained vision-language models. However, existing two-stage detection models still face two critical challenges: 1) Insufficient feature representation for novel classes, where the Region Proposal Network (RPN) struggles to generate high-quality proposals for novel objects due to weak discriminative features; 2) Decoupling of classification and localization tasks, leading to a misalignment between classification scores and localization quality. Low-quality proposals may survive Non-Maximum Suppression (NMS) due to inflated classification scores. To address these issues, we propose two key solutions: First, a Novel-Class Aware Enhancement Module (NAEM) is designed. This module synergistically enhances global semantic and local detail representations in feature maps, improving the discriminative power for novel-class object shapes. By refining novel-class proposal generation, NAEM establishes a robust foundation for subsequent classification task. Next, we propose a training-inference collaborative optimization strategy to improve the classification score accuracy for novel proposals: During training, an IoU-weighted Classification Loss is introduced to amplify supervision signals for high-IoU positive samples, enhancing the model's recognition capability for well-localized proposals. During inference, we propose Localization-weighted Classification Score that calibrates classification confidence through localization quality of proposals. Experimental results demonstrate that our method achieves 50.5% \({\text{AP}}_{50}^{\text{novel}}\) on OV-COCO and 35.9% mAPr on OV-LVIS benchmarks respectively, showing significant improvements over existing methods. Extensive comparative experiments and ablation studies confirm that the proposed method reaches the advanced level in OVOD.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Open-Vocabulary Object Detection Based on Novel-Class Aware Enhancement and Classification Optimization

  • Shuhao Liu,
  • Zhi Zhang

摘要

Open-Vocabulary Object Detection (OVOD) aims to detect objects of unseen classes through pre-trained vision-language models. However, existing two-stage detection models still face two critical challenges: 1) Insufficient feature representation for novel classes, where the Region Proposal Network (RPN) struggles to generate high-quality proposals for novel objects due to weak discriminative features; 2) Decoupling of classification and localization tasks, leading to a misalignment between classification scores and localization quality. Low-quality proposals may survive Non-Maximum Suppression (NMS) due to inflated classification scores. To address these issues, we propose two key solutions: First, a Novel-Class Aware Enhancement Module (NAEM) is designed. This module synergistically enhances global semantic and local detail representations in feature maps, improving the discriminative power for novel-class object shapes. By refining novel-class proposal generation, NAEM establishes a robust foundation for subsequent classification task. Next, we propose a training-inference collaborative optimization strategy to improve the classification score accuracy for novel proposals: During training, an IoU-weighted Classification Loss is introduced to amplify supervision signals for high-IoU positive samples, enhancing the model's recognition capability for well-localized proposals. During inference, we propose Localization-weighted Classification Score that calibrates classification confidence through localization quality of proposals. Experimental results demonstrate that our method achieves 50.5% \({\text{AP}}_{50}^{\text{novel}}\) on OV-COCO and 35.9% mAPr on OV-LVIS benchmarks respectively, showing significant improvements over existing methods. Extensive comparative experiments and ablation studies confirm that the proposed method reaches the advanced level in OVOD.