Scholarlink

Introduction <p>Manual identification of case narratives with specific relevant information can be challenging when working with large numbers of adverse event reports (case series). The process can be supported with a search engine, but building search queries often remains a manual task. Suggesting terms to add to the search query could support assessors in the identification of case narratives within a case series.</p> Objective <p>The aim of this study is to explore the feasibility of identifying case narratives containing specific characteristics with a narrative search engine supported by artificial intelligence (AI) query suggestions.</p> Methods <p>The narrative search engine uses Best Match 25 (BM25) and suggests additional query terms from two word embedding models providing English and biomedical words to a human in the loop. We calculated the percentage of relevant narratives retrieved by the system (recall) and the percentage of retrieved narratives relevant to the search (precision) on an evaluation dataset including narratives from VigiBase, the World Health Organization global database of adverse event reports for medicines and vaccines. Exact-match search and BM25 search with the Relevance Model (RM3), an alternative way to expand queries, were used as comparators.</p> Results <p>The gold standard included 55/750 narratives labelled as relevant. Our narrative search engine retrieved on average 56.4% of the relevant narratives&#xa0;(recall), which is higher when compared with exact-match search (21.8%), without a significant drop in precision &#xa0;(54.5%&#xa0;to 43.1%). The recall is also higher&#xa0;as&#xa0;compared with RM3 (34.4%).</p> Conclusions <p>Our study demonstrates that a narrative search engine supported by AI query suggestions can be a viable alternative to an exact-match search&#xa0;and BM25 search with RM3, since it can facilitate the retrieval of additional relevant narratives during signal assessments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Narrative Search Engine for Case Series Assessment Supported by Artificial Intelligence Query Suggestions

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年3月15日

查看全文
  • Alem Zekarias,
  • Eva-Lisa Meldau,
  • Shachi Bista,
  • Joana Félix China,
  • Lovisa Sandberg

摘要

Introduction

Manual identification of case narratives with specific relevant information can be challenging when working with large numbers of adverse event reports (case series). The process can be supported with a search engine, but building search queries often remains a manual task. Suggesting terms to add to the search query could support assessors in the identification of case narratives within a case series.

Objective

The aim of this study is to explore the feasibility of identifying case narratives containing specific characteristics with a narrative search engine supported by artificial intelligence (AI) query suggestions.

Methods

The narrative search engine uses Best Match 25 (BM25) and suggests additional query terms from two word embedding models providing English and biomedical words to a human in the loop. We calculated the percentage of relevant narratives retrieved by the system (recall) and the percentage of retrieved narratives relevant to the search (precision) on an evaluation dataset including narratives from VigiBase, the World Health Organization global database of adverse event reports for medicines and vaccines. Exact-match search and BM25 search with the Relevance Model (RM3), an alternative way to expand queries, were used as comparators.

Results

The gold standard included 55/750 narratives labelled as relevant. Our narrative search engine retrieved on average 56.4% of the relevant narratives (recall), which is higher when compared with exact-match search (21.8%), without a significant drop in precision  (54.5% to 43.1%). The recall is also higher as compared with RM3 (34.4%).

Conclusions

Our study demonstrates that a narrative search engine supported by AI query suggestions can be a viable alternative to an exact-match search and BM25 search with RM3, since it can facilitate the retrieval of additional relevant narratives during signal assessments.