Instance segmentation plays a vital role in practical applications, including autonomous driving and robotic vision. Among deep learning methods, query-based approaches have recently attracted significant attention due to their effectiveness in building end-to-end pipelines that eliminate complex post-processing steps. However, existing methods often fail to strike an optimal balance between accuracy and real-time performance. To address this challenge, we propose SAP-DQR, a framework integrating Spatial-Adaptive Pyramid features and Adaptive Query Reorganization to optimize the speed-accuracy trade-off. First, we design a Spatial-Adaptive Pyramid Feature Network (SAP-FPN) that integrates Asymmetric and Separable Multi-scale Pooling (ASMP) and Spatial-Channel Grouped Fusion (SCG). This design significantly accelerates and improves the efficiency of multi-scale feature processing. Additionally, we introduce a novel lightweight Global Semantic-guided Multi-stage Adaptive Query Reorganization (GSMR) mechanism that effectively provides finer-grained, higher-quality adaptive queries for various decoder stages. Our method is the first ResNet-50-based approach to surpass 40 AP on COCO while achieving real-time inference ( \(\ge \) 30 FPS), delivering an optimal speed-accuracy trade-off for latency-sensitive applications like autonomous driving. It significantly outperforms current mainstream methods, setting a new state-of-the-art for real-time instance segmentation with optimized speed-accuracy trade-off.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SAP-DQR: Joining Spatial-Adaptive Pyramid and Adaptive Query Reorganization for Speed-Accuracy Instance Segmentation

  • Jiahao Zou,
  • Congxuan Zhang,
  • Liyue Ge,
  • Chao He,
  • Jiawen Yang,
  • Zhen Chen,
  • Ke Lu

摘要

Instance segmentation plays a vital role in practical applications, including autonomous driving and robotic vision. Among deep learning methods, query-based approaches have recently attracted significant attention due to their effectiveness in building end-to-end pipelines that eliminate complex post-processing steps. However, existing methods often fail to strike an optimal balance between accuracy and real-time performance. To address this challenge, we propose SAP-DQR, a framework integrating Spatial-Adaptive Pyramid features and Adaptive Query Reorganization to optimize the speed-accuracy trade-off. First, we design a Spatial-Adaptive Pyramid Feature Network (SAP-FPN) that integrates Asymmetric and Separable Multi-scale Pooling (ASMP) and Spatial-Channel Grouped Fusion (SCG). This design significantly accelerates and improves the efficiency of multi-scale feature processing. Additionally, we introduce a novel lightweight Global Semantic-guided Multi-stage Adaptive Query Reorganization (GSMR) mechanism that effectively provides finer-grained, higher-quality adaptive queries for various decoder stages. Our method is the first ResNet-50-based approach to surpass 40 AP on COCO while achieving real-time inference ( \(\ge \) 30 FPS), delivering an optimal speed-accuracy trade-off for latency-sensitive applications like autonomous driving. It significantly outperforms current mainstream methods, setting a new state-of-the-art for real-time instance segmentation with optimized speed-accuracy trade-off.