The revision step in the Case-Based Reasoning (CBR) cycle ensures that cases are adaptable and that updates can be integrated meaningfully based on evaluation metrics. However, the effectiveness of this step heavily depends on how new knowledge is acquired to support revision. In the iSee project, where CBR is used for explanation strategy recommendations, revision knowledge is typically derived from end-user feedback following an interactive, user-centric explanation experience. This raises the research question: how can we discover this knowledge from user interactions, and how can Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) enhance the explanation experience while capturing useful knowledge for the discovery? In this paper, we propose a methodology for detecting the evolution of user explanation intent (an indicator for the need for revision) through LLM-RAG enhanced interactions. Finally, experimental results evaluate the success of the proposed methodology, assessing its validity and, in particular, the reasonability of the LLM-RAG approach. Experimental results across multiple real-world use cases demonstrate that our methodology produces highly coherent and context-aware explanations, improving overall explanation clarity, and effectively identifies when explanation strategies require revision.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Context Driven Multi-query Resolution Using LLM-RAG to Support the Revision of Explainability Needs

  • Lasal Jayawardena,
  • Anne Liret,
  • Nirmalie Wiratunga,
  • Ikechukwu Nkisi-Orji,
  • Bruno Fleisch

摘要

The revision step in the Case-Based Reasoning (CBR) cycle ensures that cases are adaptable and that updates can be integrated meaningfully based on evaluation metrics. However, the effectiveness of this step heavily depends on how new knowledge is acquired to support revision. In the iSee project, where CBR is used for explanation strategy recommendations, revision knowledge is typically derived from end-user feedback following an interactive, user-centric explanation experience. This raises the research question: how can we discover this knowledge from user interactions, and how can Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) enhance the explanation experience while capturing useful knowledge for the discovery? In this paper, we propose a methodology for detecting the evolution of user explanation intent (an indicator for the need for revision) through LLM-RAG enhanced interactions. Finally, experimental results evaluate the success of the proposed methodology, assessing its validity and, in particular, the reasonability of the LLM-RAG approach. Experimental results across multiple real-world use cases demonstrate that our methodology produces highly coherent and context-aware explanations, improving overall explanation clarity, and effectively identifies when explanation strategies require revision.