Scholarlink

<p>The generative retrieval (GR) paradigm, as a new state-of-the-art model in information retrieval, encodes a corpus of documents into its parameters and directly generates document identifiers (docids) for user queries. Despite the exciting performance, these studies failed to generalize on dynamic corpora. To bridge this research gap, we propose K-ENGINE (Katz measure-ENhanced Generative INdExer), to empower the vanilla GR model generalizing on dynamic corpora. K-ENGINE is built upon the assumption that there is an inherent correlation between documents in the corpora rather than generic semantic similarity relationships. The so-called “inherent correlations” refer to the multi-hop associations of documents within a graph structure, which arise from the transitivity of relevance. To model such associations, we introduce a novel document connectivity strategy based on the Katz measure. This strategy captures deep multi-hop connections between documents that go beyond generic semantic similarity, enabling the identification of profound inter-document associations that would be deemed irrelevant under standard cosine similarity calculations. K-ENGINE enhances the GR paradigm through two principal innovations: (1) It constructs a hierarchical semantic structure tree via spectral clustering on the Katz-based similarity matrix, which inherently captures multi-hop associations. These clusters provide richer relational priors, enabling the model to identify thematic communities through shared hierarchical prefixes, which in turn significantly improves model convergence and docid assignment; (2) Without retraining the core GR model, new documents are associated via Katz similarity using distributed parallel processing for real-time retrieval on dynamically expanding corpora, enabling efficient incremental updates. Extensive experiments conducted on the Natural Questions (NQ) dataset under batch-incremental dynamic corpus scenarios demonstrate that K-ENGINE achieves statistically significant improvements over representative generative retrieval baselines (e.g., SEAL, DSI, NCI, DSI++) and dense retrieval models (e.g., DPR). Our work establishes a GR framework that effectively handles dynamic corpora while preserving GR’s inherent efficiency advantage.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Katz measure-enhanced incremental information retrieval approach based on spectral clustering

  • 期刊论文

  • 发表日期：

    2026年3月26日

查看全文
  • Li Xiang,
  • Wenning Hao,
  • Feiyan Sun,
  • Yu Zheng,
  • Shichen Zou,
  • Ao Zou

摘要

The generative retrieval (GR) paradigm, as a new state-of-the-art model in information retrieval, encodes a corpus of documents into its parameters and directly generates document identifiers (docids) for user queries. Despite the exciting performance, these studies failed to generalize on dynamic corpora. To bridge this research gap, we propose K-ENGINE (Katz measure-ENhanced Generative INdExer), to empower the vanilla GR model generalizing on dynamic corpora. K-ENGINE is built upon the assumption that there is an inherent correlation between documents in the corpora rather than generic semantic similarity relationships. The so-called “inherent correlations” refer to the multi-hop associations of documents within a graph structure, which arise from the transitivity of relevance. To model such associations, we introduce a novel document connectivity strategy based on the Katz measure. This strategy captures deep multi-hop connections between documents that go beyond generic semantic similarity, enabling the identification of profound inter-document associations that would be deemed irrelevant under standard cosine similarity calculations. K-ENGINE enhances the GR paradigm through two principal innovations: (1) It constructs a hierarchical semantic structure tree via spectral clustering on the Katz-based similarity matrix, which inherently captures multi-hop associations. These clusters provide richer relational priors, enabling the model to identify thematic communities through shared hierarchical prefixes, which in turn significantly improves model convergence and docid assignment; (2) Without retraining the core GR model, new documents are associated via Katz similarity using distributed parallel processing for real-time retrieval on dynamically expanding corpora, enabling efficient incremental updates. Extensive experiments conducted on the Natural Questions (NQ) dataset under batch-incremental dynamic corpus scenarios demonstrate that K-ENGINE achieves statistically significant improvements over representative generative retrieval baselines (e.g., SEAL, DSI, NCI, DSI++) and dense retrieval models (e.g., DPR). Our work establishes a GR framework that effectively handles dynamic corpora while preserving GR’s inherent efficiency advantage.