There are several methods for finding set similarity and many different set similarity metrics (SSMs) have been proposed in the literature that serve specific applications. For example, the traditional SSM, such as the Jaccard index, is utilized for document deduplication in Web searches, whereas the Adamic–Adar index is used to predict links in social network analysis. However, while processing a large amount of data, it is very difficult to calculate the similarity between all possible pairs of sets involved in a given application scenario. This work aims to analyze efficient estimators for set similarity metrics. Firstly, the most relevant SSMs are introduced to understand the actual semantics of designing efficient estimators for these SSMs. Secondly, various estimators for SSMs (such as MinHash, SimHash, DotHash) are analyzed and summarized for state of the art. Finally, we have performed several experiments for link prediction tasks with these estimators by using real-world datasets: Facebook, Wikipedia, and Ogbl-ddi. Empirical results demonstrate that the DotHash estimator performs better than other estimators for link prediction tasks based on the Hits@K metric.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Estimating Set Similarity Matrices for Link Prediction: An Empirical Analysis

  • Alankrita Mingwal,
  • Vijay Verma

摘要

There are several methods for finding set similarity and many different set similarity metrics (SSMs) have been proposed in the literature that serve specific applications. For example, the traditional SSM, such as the Jaccard index, is utilized for document deduplication in Web searches, whereas the Adamic–Adar index is used to predict links in social network analysis. However, while processing a large amount of data, it is very difficult to calculate the similarity between all possible pairs of sets involved in a given application scenario. This work aims to analyze efficient estimators for set similarity metrics. Firstly, the most relevant SSMs are introduced to understand the actual semantics of designing efficient estimators for these SSMs. Secondly, various estimators for SSMs (such as MinHash, SimHash, DotHash) are analyzed and summarized for state of the art. Finally, we have performed several experiments for link prediction tasks with these estimators by using real-world datasets: Facebook, Wikipedia, and Ogbl-ddi. Empirical results demonstrate that the DotHash estimator performs better than other estimators for link prediction tasks based on the Hits@K metric.