Large language models (LLMs) are increasingly being deployed for a variety of tasks, including recommendation systems. They are uniquely suited to making inferences in zero-shot circumstances given a purported capacity to reason across domains. However, there are several drawbacks to their use that limit real-world applicability. Namely, LLMs make opaque judgements that are not easily understood by users or subject to human control. Additionally, when many alternatives are evaluated, a robust external procedure is needed to control the model’s predictions. The Best-Worst Method (BWM) is an MCDA technique that can be used to extract preference weights and evaluate alternatives in a pairwise manner. This work proposes a hybrid model using BWM and LLMs that provides a human-interpretable framework for AI-ML recommendation. The system can function independently or as a refinement layer on top of existing retrieval systems. Due to the use of natural language as a mediator, it can process a broad spectrum of user and alternative information. An evaluation is conducted using the MovieLens dataset, showing a positive monotonic relationship between our predicted scores and user-submitted reviews.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Synergistic Integration Between Large Language Models and the Best-Worst Method

  • Hunter Briegel,
  • Tharita Tipdecho

摘要

Large language models (LLMs) are increasingly being deployed for a variety of tasks, including recommendation systems. They are uniquely suited to making inferences in zero-shot circumstances given a purported capacity to reason across domains. However, there are several drawbacks to their use that limit real-world applicability. Namely, LLMs make opaque judgements that are not easily understood by users or subject to human control. Additionally, when many alternatives are evaluated, a robust external procedure is needed to control the model’s predictions. The Best-Worst Method (BWM) is an MCDA technique that can be used to extract preference weights and evaluate alternatives in a pairwise manner. This work proposes a hybrid model using BWM and LLMs that provides a human-interpretable framework for AI-ML recommendation. The system can function independently or as a refinement layer on top of existing retrieval systems. Due to the use of natural language as a mediator, it can process a broad spectrum of user and alternative information. An evaluation is conducted using the MovieLens dataset, showing a positive monotonic relationship between our predicted scores and user-submitted reviews.