This study aims to develop a system that allows humans to interact naturally with unique characters appearing in two-dimensional manga-style animation works. We define the manga-anime style in Japanese Anime, analyze the characteristics of the expression, and design a system that allows a dialogue agent to use Large Language Models (LLMs) to behave like a manga-anime character. The proposed system uses LLMs to 1) process Voice Activity Projection (VAP) for user speech, switch character prompts in real-time, and generate agent speech that reflects personality and context; 2) determine the graphical/sound expressions of the manga-anime style, including comic symbols, backgrounds, effects, and composition, according to the speech of the agent; and 3) generate parameters based on the content and emotion as well as link them with animations of objects that characterize the agent. Using this system, we created an artificial intelligence character named “Kohane,” whose emotional expression changes depending on the conversation with a human.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Development of a Dialogue Agent System Expressed in a Two-Dimensional Manga-Anime Style and Design of the Original Artificial Intelligence Character “Kohane”

  • Kaimu Harada,
  • Sachiko Kodama

摘要

This study aims to develop a system that allows humans to interact naturally with unique characters appearing in two-dimensional manga-style animation works. We define the manga-anime style in Japanese Anime, analyze the characteristics of the expression, and design a system that allows a dialogue agent to use Large Language Models (LLMs) to behave like a manga-anime character. The proposed system uses LLMs to 1) process Voice Activity Projection (VAP) for user speech, switch character prompts in real-time, and generate agent speech that reflects personality and context; 2) determine the graphical/sound expressions of the manga-anime style, including comic symbols, backgrounds, effects, and composition, according to the speech of the agent; and 3) generate parameters based on the content and emotion as well as link them with animations of objects that characterize the agent. Using this system, we created an artificial intelligence character named “Kohane,” whose emotional expression changes depending on the conversation with a human.