This chapter examines multimodality in the context of explaining AI agent behavior and reasoning. Both embodied (like robots) and unembodied (like Alexa, including text-based systems) AI systems require effective explanatory mechanisms to help users understand their functions and decisions. The analysis reveals striking similarities between design elements used for encouraging user collaboration with AI systems and those employed in creating multimodal explanations. Despite the considerable advantages offered by multimodality-based sXAI, these design choices may produce unintended negative effects on users receiving the explanations. To address this challenge, the chapter draws on philosophical research regarding emotion co-construction. By integrating methodological approaches and theoretical insights from the philosophy of science with current sXAI studies, designers can create explanations that emerge collaboratively between the explainer and explainee. Such explanations recognize how emotions influence both how humans think and how they make decisions. The path toward more effective multimodal explanations requires recognizing how users’ emotional responses shape their understanding of AI systems. Through careful design considerations, explanatory systems can acknowledge this emotional dimension while still delivering technically accurate information about AI behavior.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodality in Agents

  • Rachele Carli,
  • Sukriti Bhattacharya,
  • Igor Tchappi,
  • Kary Främling,
  • Amro Najjar

摘要

This chapter examines multimodality in the context of explaining AI agent behavior and reasoning. Both embodied (like robots) and unembodied (like Alexa, including text-based systems) AI systems require effective explanatory mechanisms to help users understand their functions and decisions. The analysis reveals striking similarities between design elements used for encouraging user collaboration with AI systems and those employed in creating multimodal explanations. Despite the considerable advantages offered by multimodality-based sXAI, these design choices may produce unintended negative effects on users receiving the explanations. To address this challenge, the chapter draws on philosophical research regarding emotion co-construction. By integrating methodological approaches and theoretical insights from the philosophy of science with current sXAI studies, designers can create explanations that emerge collaboratively between the explainer and explainee. Such explanations recognize how emotions influence both how humans think and how they make decisions. The path toward more effective multimodal explanations requires recognizing how users’ emotional responses shape their understanding of AI systems. Through careful design considerations, explanatory systems can acknowledge this emotional dimension while still delivering technically accurate information about AI behavior.