Large Language Models (LLMs) are increasingly being integrated into various applications across different fields. Therefore, it is crucial to ensure that their responses or outputs are not exploited by malicious attackers. However, while previous work has revealed and addressed the issue of offensive or toxic responses, the reasoning ability of cant or dark jargon has yet to be investigated. Exploring this capability is invaluable and necessary since it can prevent cybercriminals from bypassing filters or restrictions by exploiting innocent-looking terms. In response to this challenge, this paper presents the first domain-specific Cant dataset and proposes the first evaluation framework, CantCounter, which has a four-stage strategy: Fine-Tuning, Co-Tuning, Data-Diffusion, and Data-Analysis. Through comprehensive experiments in the real world, we discover that cant can bypass filters in state-of-the-art LLMs such as ChatGPT, and we find different recognition accuracy based on different question types, question setups, and prompt clues. We also demonstrate that among these LLMs, more recently updated models exhibit a lower probability of refusing to answer the cant questions. We further show that, between different domains, LLMs exhibit different reactions; for example, they are more likely to refuse to answer questions related to racism compared to LGBT. Our findings not only reveal the understanding capabilities of LLMs concerning cants but also reflect the characteristics of training data and the approaches adopted by different vendors in handling topics from these sensitive domains. We make our datasets and code available at https://github.com/BESTICSP/CantCounter .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Can’t Say Cant? Measuring and Reasoning of Dark Jargons in Large Language Models

  • Xu Ji,
  • Jianyi Zhang,
  • Ziyin Zhou,
  • Zhangchi Zhao,
  • Qianqian Qiao,
  • Kaiying Han,
  • Md Imran Hossen,
  • Xiali Hei

摘要

Large Language Models (LLMs) are increasingly being integrated into various applications across different fields. Therefore, it is crucial to ensure that their responses or outputs are not exploited by malicious attackers. However, while previous work has revealed and addressed the issue of offensive or toxic responses, the reasoning ability of cant or dark jargon has yet to be investigated. Exploring this capability is invaluable and necessary since it can prevent cybercriminals from bypassing filters or restrictions by exploiting innocent-looking terms. In response to this challenge, this paper presents the first domain-specific Cant dataset and proposes the first evaluation framework, CantCounter, which has a four-stage strategy: Fine-Tuning, Co-Tuning, Data-Diffusion, and Data-Analysis. Through comprehensive experiments in the real world, we discover that cant can bypass filters in state-of-the-art LLMs such as ChatGPT, and we find different recognition accuracy based on different question types, question setups, and prompt clues. We also demonstrate that among these LLMs, more recently updated models exhibit a lower probability of refusing to answer the cant questions. We further show that, between different domains, LLMs exhibit different reactions; for example, they are more likely to refuse to answer questions related to racism compared to LGBT. Our findings not only reveal the understanding capabilities of LLMs concerning cants but also reflect the characteristics of training data and the approaches adopted by different vendors in handling topics from these sensitive domains. We make our datasets and code available at https://github.com/BESTICSP/CantCounter .