Companies with long operational histories often face the challenge of managing vast repositories of documentation, which hold critical knowledge needed for maintaining ongoing projects. Retrieving relevant information from these extensive archives is a time-consuming and complex task, requiring specialized expertise and familiarity with outdated terminology. Semantic search has emerged as a promising technology to address these issues by improving the precision and efficiency of information retrieval. In this paper, we present our collaborative research with Hitachi Energy, exploring the development of a semantic search engine based on existing open-source solutions to assist practitioners in searching large industrial historical document repositories. We first analyzed available No-SQL databases with search-engine interfaces, followed by an evaluation of pre-trained semantic transformers to determine which offers the best balance of accuracy and speed for semantic search. Our research identified OpenSearch as the most suitable No-SQL database due to its flexibility, free usage, and support for semantic transformers. After evaluating various pre-trained semantic transformers, we found all-MiniLM-L6-v2 to offer the best balance of accuracy and speed for semantic search. Based on the findings, we developed a prototype AI-powered semantic search tool, which was tested in a workshop involving Hitachi Energy professionals. Our findings demonstrate the feasibility and effectiveness of AI-powered semantic search for handling historical documentation, offering significant potential for industries tasked with managing large legacy archives.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

AI-Powered Semantic Search for Historical Documentation: A Collaborative Research with Hitachi Energy

  • Ivan Kelly Maranan Hansson,
  • Edvin Wiklund,
  • Alessio Bucaioni,
  • Luciana Provenzano

摘要

Companies with long operational histories often face the challenge of managing vast repositories of documentation, which hold critical knowledge needed for maintaining ongoing projects. Retrieving relevant information from these extensive archives is a time-consuming and complex task, requiring specialized expertise and familiarity with outdated terminology. Semantic search has emerged as a promising technology to address these issues by improving the precision and efficiency of information retrieval. In this paper, we present our collaborative research with Hitachi Energy, exploring the development of a semantic search engine based on existing open-source solutions to assist practitioners in searching large industrial historical document repositories. We first analyzed available No-SQL databases with search-engine interfaces, followed by an evaluation of pre-trained semantic transformers to determine which offers the best balance of accuracy and speed for semantic search. Our research identified OpenSearch as the most suitable No-SQL database due to its flexibility, free usage, and support for semantic transformers. After evaluating various pre-trained semantic transformers, we found all-MiniLM-L6-v2 to offer the best balance of accuracy and speed for semantic search. Based on the findings, we developed a prototype AI-powered semantic search tool, which was tested in a workshop involving Hitachi Energy professionals. Our findings demonstrate the feasibility and effectiveness of AI-powered semantic search for handling historical documentation, offering significant potential for industries tasked with managing large legacy archives.