Entity resolution (ER) is essential in data processing, aiming to identify and merge records representing the same real-world entities accurately. This task is critical for ensuring data quality and consistency across various applications. This study applies a Transformer-based Denoising Adversarial Variational approach to a synthetic dataset like DLBP-ACM, iTunes-AMAZON, etc. Featuring diverse entity records. This synthetic dataset is specifically designed to challenge and validate the robustness of the ER model by applying noise, duplication, and heterogeneity. This model uses the Sentence Embedding Module to generate semantic information from the simulated dataset. Then, it uses the adversarial autoencoders to convert low-level embeddings into high-level probability distributions. Using these high-level probability distributions, a supervised entity matching model is deployed to determine if the two records represent the same entity. We compare this method with ER approaches, like Deep Match, Deep Entity Matching with Pre-trained Language Model(DITTO), etc., to measure their performance against this model. Ultimately, we apply an explainable AI component to the model to identify the attributes used mainly by the model, which will help tweak the model hyperparameters if needed.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Entity Resolution Using Transformers for Synthetic Datasets

  • Md Rizwanul Kabir,
  • Aatif Mohammad Althaf,
  • Md Samin Morshed,
  • Mariofanna Milanova,
  • John Talburt

摘要

Entity resolution (ER) is essential in data processing, aiming to identify and merge records representing the same real-world entities accurately. This task is critical for ensuring data quality and consistency across various applications. This study applies a Transformer-based Denoising Adversarial Variational approach to a synthetic dataset like DLBP-ACM, iTunes-AMAZON, etc. Featuring diverse entity records. This synthetic dataset is specifically designed to challenge and validate the robustness of the ER model by applying noise, duplication, and heterogeneity. This model uses the Sentence Embedding Module to generate semantic information from the simulated dataset. Then, it uses the adversarial autoencoders to convert low-level embeddings into high-level probability distributions. Using these high-level probability distributions, a supervised entity matching model is deployed to determine if the two records represent the same entity. We compare this method with ER approaches, like Deep Match, Deep Entity Matching with Pre-trained Language Model(DITTO), etc., to measure their performance against this model. Ultimately, we apply an explainable AI component to the model to identify the attributes used mainly by the model, which will help tweak the model hyperparameters if needed.