Scholarlink

<p>RGB-Thermal (RGB-T) object detection in Unmanned Aerial Vehicle (UAV) imagery typically relies on fusing multimodal information to detect multiple objects under all-weather conditions. However, existing fusion methods primarily rely on pairwise attention mechanisms, thereby overlooking group-wise dependencies. Such oversight renders these methods ineffective at capturing collective relationships among objects in complex aerial scenes, including vehicle queues or cases of heavy occlusion. Additionally, the densely connected structures in these fusion methods often result in high computational costs. To address these limitations, we propose a lightweight cross-modal hypergraph fusion for RGBT object detection, termed HyperDet. The core of HyperDet is an Adaptive Hierarchical Multi-Modal Fusion (AHMF) framework, which follows a cascaded enhancement-reasoning paradigm. Specifically, AHMF first employs a Dual Cross-Modal Feature Alignment (DCFA) module to decouple and align dense RGB textures and sparse thermal signatures, producing a calibrated feature space. On top of this aligned representation, a Cross-Modal Adaptive Hypergraph (CAH) module is introduced to perform high-order reasoning. Unlike pairwise approaches, CAH constructs dynamic hyperedges that aggregate spatially dispersed but semantically consistent instances across modalities. This mechanism enables the model to explicitly capture long-range, group-wise dependencies among objects, allowing for robust recognition of complex clusters. Extensive experiments on three RGBT object detection benchmarks, i.e., DroneVehicle, VEDAI, and LLVIP, demonstrate that HyperDet achieves state-of-the-art performance. For example, it achieves 86.4% mAP<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(_{50}\)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mn>50</mn> <mrow /> </mmultiscripts> </math></EquationSource> </InlineEquation> on DroneVehicle with only 3.2 M parameters. The code for HyperDet is available at <a href="https://github.com/lihui-ioe/HyperDet">https://github.com/lihui-ioe/HyperDet</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

HyperDet: Cross-modal hypergraph fusion for enhanced object detection in UAV imagery

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月28日

查看全文
  • Hui Li,
  • Jianbo Ma,
  • Jianlin Zhang,
  • Hui Luo,
  • Haorui Zuo,
  • Yunhao Wu,
  • Tong Tan

摘要

RGB-Thermal (RGB-T) object detection in Unmanned Aerial Vehicle (UAV) imagery typically relies on fusing multimodal information to detect multiple objects under all-weather conditions. However, existing fusion methods primarily rely on pairwise attention mechanisms, thereby overlooking group-wise dependencies. Such oversight renders these methods ineffective at capturing collective relationships among objects in complex aerial scenes, including vehicle queues or cases of heavy occlusion. Additionally, the densely connected structures in these fusion methods often result in high computational costs. To address these limitations, we propose a lightweight cross-modal hypergraph fusion for RGBT object detection, termed HyperDet. The core of HyperDet is an Adaptive Hierarchical Multi-Modal Fusion (AHMF) framework, which follows a cascaded enhancement-reasoning paradigm. Specifically, AHMF first employs a Dual Cross-Modal Feature Alignment (DCFA) module to decouple and align dense RGB textures and sparse thermal signatures, producing a calibrated feature space. On top of this aligned representation, a Cross-Modal Adaptive Hypergraph (CAH) module is introduced to perform high-order reasoning. Unlike pairwise approaches, CAH constructs dynamic hyperedges that aggregate spatially dispersed but semantically consistent instances across modalities. This mechanism enables the model to explicitly capture long-range, group-wise dependencies among objects, allowing for robust recognition of complex clusters. Extensive experiments on three RGBT object detection benchmarks, i.e., DroneVehicle, VEDAI, and LLVIP, demonstrate that HyperDet achieves state-of-the-art performance. For example, it achieves 86.4% mAP \(_{50}\) 50 on DroneVehicle with only 3.2 M parameters. The code for HyperDet is available at https://github.com/lihui-ioe/HyperDet.