Measuring distance and similarity is one of the most fundamental problems in data science and statistics. Proximity functions are used almost everywhere, from clustering problems to outlier detection, database systems, and data privacy. One important challenge is how to handle heterogeneous data, which makes up a large part of real-world tabular data. Euclidean distance with dummy variables is only half a solution, and Gower’s similarity, which is usually the go-to choice for similarity estimation in heterogeneous data, also lacks stability when it comes to combinations of categorical variables. In this paper, we propose Resample Exposure, as a novel heterogeneous measure that effectively captures the similarity of records based on resampling probability and distance with traversal penalty. Resample exposure integrates well with various distance- or similarity-based tasks; we show that the measure is competitive for nearest neighbour classification and viable for partitioning around medoids, making resample exposure a relevant measure for heterogeneous data applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Similarity Based on Resample Exposure

  • Anton Danholt Lautrup,
  • Hafiz Saud Arshad,
  • Tobias Hyrup,
  • Muhammad Rajabinasab,
  • Arthur Zimek,
  • Peter Schneider-Kamp

摘要

Measuring distance and similarity is one of the most fundamental problems in data science and statistics. Proximity functions are used almost everywhere, from clustering problems to outlier detection, database systems, and data privacy. One important challenge is how to handle heterogeneous data, which makes up a large part of real-world tabular data. Euclidean distance with dummy variables is only half a solution, and Gower’s similarity, which is usually the go-to choice for similarity estimation in heterogeneous data, also lacks stability when it comes to combinations of categorical variables. In this paper, we propose Resample Exposure, as a novel heterogeneous measure that effectively captures the similarity of records based on resampling probability and distance with traversal penalty. Resample exposure integrates well with various distance- or similarity-based tasks; we show that the measure is competitive for nearest neighbour classification and viable for partitioning around medoids, making resample exposure a relevant measure for heterogeneous data applications.