Despite the large amount of data available within and beyond open data portals, most of them remain “dark data” - unused. A significant factor contributing to this is the usability challenges, such as poor data findability and discoverability. These challenges can be partly addressed through a recommendation system that suggests datasets related to those of users’ interests. Unlike other domains, a recommendation system in open data portals cannot rely on user profiles and associated historical data, since most open data portals do not require authentication. As such, existing recommendation systems for open data portals mostly focus on tag- and keywords-based recommendations or allow users to navigate to datasets of potential interest through categories they belong to, thereby failing to capture the semantic meaning of dataset metadata when making recommendations. This study proposes a dataset recommendation method -TDA- that uses datasets’ metadata capturing its semantic meaning, thereby providing users with semantically rich context-aware and as such more meaningful recommendations of related datasets. To capture semantic relations between dataset metadata, the proposed recommendation system leverages Natural Language Processing using pre-trained GloVe embeddings via SpaCy. The metadata elements used as an input by the recommender are selected to make it compatible with a wider range of open data portals, keeping the overall architecture of the prototype simple and lightweight. Preliminary usability testing conducted employing User Acceptance Testing (UAT) demonstrates its potential to improve data discoverability, with the collected feedback informing its refinement.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

May the Data Be with You: Towards an AI-Powered Semantic Recommender for Unlocking Dark Data

  • Ramil Huseynov,
  • Anastasija Nikiforova,
  • Dimitrios Symeonidis,
  • David Duenas-Cid

摘要

Despite the large amount of data available within and beyond open data portals, most of them remain “dark data” - unused. A significant factor contributing to this is the usability challenges, such as poor data findability and discoverability. These challenges can be partly addressed through a recommendation system that suggests datasets related to those of users’ interests. Unlike other domains, a recommendation system in open data portals cannot rely on user profiles and associated historical data, since most open data portals do not require authentication. As such, existing recommendation systems for open data portals mostly focus on tag- and keywords-based recommendations or allow users to navigate to datasets of potential interest through categories they belong to, thereby failing to capture the semantic meaning of dataset metadata when making recommendations. This study proposes a dataset recommendation method -TDA- that uses datasets’ metadata capturing its semantic meaning, thereby providing users with semantically rich context-aware and as such more meaningful recommendations of related datasets. To capture semantic relations between dataset metadata, the proposed recommendation system leverages Natural Language Processing using pre-trained GloVe embeddings via SpaCy. The metadata elements used as an input by the recommender are selected to make it compatible with a wider range of open data portals, keeping the overall architecture of the prototype simple and lightweight. Preliminary usability testing conducted employing User Acceptance Testing (UAT) demonstrates its potential to improve data discoverability, with the collected feedback informing its refinement.