The goal of role-playing is to ensure that LLMs generate responses that fully embody the specified character, adhering to the role’s background, speaking style, personality, and typical behaviors. Various methods aim to improve LLMs’ character simulation capabilities, such as leveraging specialized prompts and fine-tuning with character dialogues. However, no research has yet investigated the relationship between LLMs’ capabilities and their role-playing performance. Understanding this relationship could provide insights into selecting the most suitable base model for mimicking realistic characters and improving LLM performance from the outset of the task. In this paper, we investigate three main factors influencing role-playing performance: LLMs’ personalities and those of the target characters, LLMs’ familiarity with the character, and their conversational abilities. We conduct experiments with 11 models (including both prompt-based and fine-tuning methods) across 7 characters with varying personalities and backgrounds. The results indicate that both LLMs’ and characters’ personalities are primary factors affecting performance, while insufficient familiarity leads to poor outcomes. Importantly, we find no evidence that sharing the same personality with a character directly correlates with enhanced role-playing performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Acting Technique: Factors that Influence the Effectiveness of Role-Playing Model

  • Baohua Zhang,
  • Yongyi Huang,
  • WenYao Cui,
  • Huaping Zhang

摘要

The goal of role-playing is to ensure that LLMs generate responses that fully embody the specified character, adhering to the role’s background, speaking style, personality, and typical behaviors. Various methods aim to improve LLMs’ character simulation capabilities, such as leveraging specialized prompts and fine-tuning with character dialogues. However, no research has yet investigated the relationship between LLMs’ capabilities and their role-playing performance. Understanding this relationship could provide insights into selecting the most suitable base model for mimicking realistic characters and improving LLM performance from the outset of the task. In this paper, we investigate three main factors influencing role-playing performance: LLMs’ personalities and those of the target characters, LLMs’ familiarity with the character, and their conversational abilities. We conduct experiments with 11 models (including both prompt-based and fine-tuning methods) across 7 characters with varying personalities and backgrounds. The results indicate that both LLMs’ and characters’ personalities are primary factors affecting performance, while insufficient familiarity leads to poor outcomes. Importantly, we find no evidence that sharing the same personality with a character directly correlates with enhanced role-playing performance.