<p>The security of agents based on large language models (LLMs) has received increasing attention in recent years. Jailbreak attacks on agents can bypass security restrictions, uncover potential vulnerabilities, and thus promote research in agent security. However, existing agent jailbreak methods face several limitations: (1) reliance on static rules and insufficient adaptability to dynamic environments; (2) lack of systematic modeling of memory vulnerabilities in complex workflows; (3) evaluation frameworks that overlook long-term concealment and lateral penetration capabilities; (4) incomplete target agents in jailbreak attacks. To address these issues, we propose an empirical jailbreak method: CPMP, which combines Context Perturbation and Memory Poisoning to attack the agent’s short-term and long-term memory simultaneously. Additionally, we introduce SCA (Smart Customer Agent) for e-commerce customer service dialogues, consisting of five complete modules. The jailbreak method dynamically adjusts its attack strategy across multiple interaction dialogues, increasing both the stealthiness and complexity of the attack. Through experiments, we demonstrate the effectiveness of CPMP and showcase its powerful attack capability in e-commerce customer service scenarios.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Exploring the security of short-term and long-term contextual memory in industrial smart customer agent

  • Yanxu Mao,
  • Tiehan Cui,
  • Jinyang Wang,
  • Datao You

摘要

The security of agents based on large language models (LLMs) has received increasing attention in recent years. Jailbreak attacks on agents can bypass security restrictions, uncover potential vulnerabilities, and thus promote research in agent security. However, existing agent jailbreak methods face several limitations: (1) reliance on static rules and insufficient adaptability to dynamic environments; (2) lack of systematic modeling of memory vulnerabilities in complex workflows; (3) evaluation frameworks that overlook long-term concealment and lateral penetration capabilities; (4) incomplete target agents in jailbreak attacks. To address these issues, we propose an empirical jailbreak method: CPMP, which combines Context Perturbation and Memory Poisoning to attack the agent’s short-term and long-term memory simultaneously. Additionally, we introduce SCA (Smart Customer Agent) for e-commerce customer service dialogues, consisting of five complete modules. The jailbreak method dynamically adjusts its attack strategy across multiple interaction dialogues, increasing both the stealthiness and complexity of the attack. Through experiments, we demonstrate the effectiveness of CPMP and showcase its powerful attack capability in e-commerce customer service scenarios.