Scholarlink

Taxonomies, structured as tree hierarchies, are valuable for applications such as web retrieval, question-answering, and recommender systems. As existing taxonomy curation based on deep learning or pre-trained models rely heavily on a large amount of labeled data, which is extremely time-consuming and labor-intensive, the emergence of large language models (LLMs, e.g., ChatGPT, GPT-4.0, LLaMA2) has made automatic taxonomy construction from texts highly desirable. However, relying solely on LLMs and prompt engineering makes it challenging to extract precise and comprehensive is-a relationships from texts. In response to this limitation, this paper aims to explore a well-designed approach to guide LLMs for better construction of taxonomies from texts. On one hand, we propose a doubly-checked mechanism to improve the quality of candidate nodes generated from texts. On the other hand, we utilize a modularized Chain-of-Thought prompting technique to break down hypernym identification into several sub-problems and employ the beam search-based self-evaluation strategy to enhance reliability. Specifically, self-evaluation constraint factors are introduced to score the reliability of reasoning chains generated through beam search, whereby selecting the most reliable chain as the final judgment. Extensive experiments on three datasets from various domains verify that the proposed method significantly improves the performance of taxonomy induction from texts.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Taxonomy Induction Using LLMs: An Enhanced Framework by Integrating Doubly-Checked Mechanism and Self-evaluation Strategy

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Jiaye Li,
  • Yuan Meng,
  • Lijun Wang,
  • Tianhao Qian,
  • Songlin Zhai,
  • Guilin Qi

摘要

Taxonomies, structured as tree hierarchies, are valuable for applications such as web retrieval, question-answering, and recommender systems. As existing taxonomy curation based on deep learning or pre-trained models rely heavily on a large amount of labeled data, which is extremely time-consuming and labor-intensive, the emergence of large language models (LLMs, e.g., ChatGPT, GPT-4.0, LLaMA2) has made automatic taxonomy construction from texts highly desirable. However, relying solely on LLMs and prompt engineering makes it challenging to extract precise and comprehensive is-a relationships from texts. In response to this limitation, this paper aims to explore a well-designed approach to guide LLMs for better construction of taxonomies from texts. On one hand, we propose a doubly-checked mechanism to improve the quality of candidate nodes generated from texts. On the other hand, we utilize a modularized Chain-of-Thought prompting technique to break down hypernym identification into several sub-problems and employ the beam search-based self-evaluation strategy to enhance reliability. Specifically, self-evaluation constraint factors are introduced to score the reliability of reasoning chains generated through beam search, whereby selecting the most reliable chain as the final judgment. Extensive experiments on three datasets from various domains verify that the proposed method significantly improves the performance of taxonomy induction from texts.