Detecting Propensity Score Shifts Across Groups in Positive–Unlabeled Data
摘要
Positive and Unlabeled (PU) data arise in many real-world settings where only a subset of positive cases are labeled, while the rest, including both unlabeled positives and true negatives, remain unannotated. This situation is common in domains such as self-reported health events, adverse drug effect reporting, and medical image annotation. A key element required for PU data analysis is modeling the labeling mechanism, which is described by the propensity score function, defined as the probability of labeling a positive observation. Estimating the propensity score is a significant challenge, especially when the labeling mechanism depends on the characteristics of individual instances. In this work, we focus on heterogeneous PU data composed of two groups that may differ in their labeling mechanisms due to external factors such as interventions or annotator differences. We propose a statistical framework to test whether the propensity score functions are equal across groups. Our approach consists of a two-step procedure: a preliminary comparison of labeling proportions, followed by a novel test for the equality of conditional distributions based on McFadden’s pseudo- \(R^2\) . Through simulation experiments, we demonstrate that our method reliably detects shifts in labeling behavior, offering an effective tool for analyzing group-level heterogeneity in PU data.