<p>The unprecedented advancements in multimodal large language models (MLLMs) have demonstrated strong potential for language-vision interaction in remote sensing tasks such as visual question answering and scene understanding. However, these models are largely constrained to basic instruction-following or descriptive tasks, and struggle with real-world remote sensing scenarios where multi-source data, fine-grained spatial semantics, and expert knowledge are indispensable. To address these limitations, we propose RS-Agent, a domain-adapted intelligent agent that bridges user intent and professional remote sensing workflows through structured task planning and tool orchestration. RS-Agent is built upon four key components that explicitly follow the typical workflow of remote sensing applications: an LLM-based Central Controller for user intent understanding and analytical process planning, a dynamic toolkit for tool execution, a Solution Space for task-specific expert guidance, and a Knowledge Space for domain-level knowledge support. To further enhance the performance of RS-Agent, we introduce two novel mechanisms: Task-Aware Retrieval, which improves task planning by explicitly inferring task types and retrieving expert-defined procedural solutions, rather than relying on query-level similarity or ad-hoc tool chaining; and DualRAG, a weighted dual-path retrieval-augmented generation method, which enhances the relevance and completeness of retrieved domain knowledge. RS-Agent natively supports multiple imaging modalities, including optical and SAR imagery. For SAR tasks in particular, the agent plans and orchestrates dedicated SAR processing and analysis tools into executable workflows, improving reliability and automation under underspecified requests. Extensive experiments across 9 datasets and 18 remote sensing tasks demonstrate that RS-Agent significantly outperforms state-of-the-art MLLMs, achieving over 95% task planning accuracy and delivering superior performance in tasks such as scene classification, object counting, and remote sensing visual question answering. These results validate the effectiveness of a dedicated remote sensing agent that fuses LLM reasoning with domain expertise for geospatial intelligence. Our work presents RS-Agent as a robust and extensible framework for advancing intelligent automation in remote sensing analysis. Our code will be available at <a href="https://github.com/IntelliSensing/RS-Agent">https://github.com/IntelliSensing/RS-Agent</a>.</p>

RS-Agent: automating remote sensing tasks through intelligent agent

期刊论文
发表日期：

2026年7月27日

查看全文

Wenjia Xu,
Zijian Yu,
Boyang Mu,
Jiuniu Wang,
Zhiwei Wei,
Mugen Peng

摘要

The unprecedented advancements in multimodal large language models (MLLMs) have demonstrated strong potential for language-vision interaction in remote sensing tasks such as visual question answering and scene understanding. However, these models are largely constrained to basic instruction-following or descriptive tasks, and struggle with real-world remote sensing scenarios where multi-source data, fine-grained spatial semantics, and expert knowledge are indispensable. To address these limitations, we propose RS-Agent, a domain-adapted intelligent agent that bridges user intent and professional remote sensing workflows through structured task planning and tool orchestration. RS-Agent is built upon four key components that explicitly follow the typical workflow of remote sensing applications: an LLM-based Central Controller for user intent understanding and analytical process planning, a dynamic toolkit for tool execution, a Solution Space for task-specific expert guidance, and a Knowledge Space for domain-level knowledge support. To further enhance the performance of RS-Agent, we introduce two novel mechanisms: Task-Aware Retrieval, which improves task planning by explicitly inferring task types and retrieving expert-defined procedural solutions, rather than relying on query-level similarity or ad-hoc tool chaining; and DualRAG, a weighted dual-path retrieval-augmented generation method, which enhances the relevance and completeness of retrieved domain knowledge. RS-Agent natively supports multiple imaging modalities, including optical and SAR imagery. For SAR tasks in particular, the agent plans and orchestrates dedicated SAR processing and analysis tools into executable workflows, improving reliability and automation under underspecified requests. Extensive experiments across 9 datasets and 18 remote sensing tasks demonstrate that RS-Agent significantly outperforms state-of-the-art MLLMs, achieving over 95% task planning accuracy and delivering superior performance in tasks such as scene classification, object counting, and remote sensing visual question answering. These results validate the effectiveness of a dedicated remote sensing agent that fuses LLM reasoning with domain expertise for geospatial intelligence. Our work presents RS-Agent as a robust and extensible framework for advancing intelligent automation in remote sensing analysis. Our code will be available at https://github.com/IntelliSensing/RS-Agent.