A framework for building explainable collaborative multimodal dialogue systems using a theory of mind
摘要
Eva is a multimodal conversational framework for building planning-based systems that help users accomplish their domain goals through collaborative dialogue. We argue that planning-based systems can do this by inferring users’ intentions, adopting goals to achieve them, developing plans to achieve those goals, detecting whether obstacles are present, finding plans to overcome them, and planning their actions, including speech acts. In the Eva framework, conversational agents can maintain and reason with their own beliefs, goals and intentions, and explicitly reason about those of their users. Belief reasoning is accomplished with a modal Horn-clause meta-interpreter. The planning and reasoning subsystems obey the principles of persistent goals and intentions, including the formation and decomposition of intentions to perform complex actions, as well as the conditions under which they can be given up. In virtue of its planning process, Eva treats its speech acts just like its other actions – physical acts affect physical states, digital acts affect digital states, and speech acts affect mental and social states. This general framework enables systems to plan a variety of speech acts including requests, informs, questions, confirmations, recommendations, offers, acceptances, greetings, and emotive expressions. Each of these has a formally specified semantics which is used during the planning and reasoning processes. Because Eva-based agents can keep track of different users’ mental states, they can engage in multi-party dialogues. Importantly, the framework supports systems’ explanations of their actions because they have created plans standing behind each of them. As a reaction to the near-universal focus on using large language models for every application, a trend has emerged recently towards integrated neuro-symbolic architectures. The Eva framework is an example of such an architecture in the area of collaborative dialogue systems.