Prompt Infection: LLM-to-LLM Prompt Injection within Multi-agent Systems
摘要
As Large Language Models (LLMs) become increasingly powerful, multi-agent systems (MAS) are widely used in modern AI applications. However, these systems pose unique security risks. In this paper, we expose an even more severe attack vector: LLM-to-LLM prompt injection, which completely overrides the system and grants an attacker full control. We introduce prompt infection, a novel attack in which malicious prompts self-replicate across interconnected agents, behaving much like a computer virus. This attack extends traditional prompt injection by introducing data propagation and self-replication, enabling controlled infection and communication between agents. Prompt Infection poses severe threats, including data theft, scams, misinformation, and system-wide disruption, all while propagating autonomously. We show that repurposing existing prompt injection defenses can help mitigate infection spread and reduce system vulnerability to prompt infection. This work underscores the urgent need to address prompt injection vulnerabilities in MAS as their adoption accelerates.