DPERC: Direct Parameter Estimation for Mixed Data with Random Missingness
摘要
The covariance matrix is a foundation in numerous statistical and machine learning applications, such as Principal Component Analysis, Correlation Heatmap, etc. However, missing values within datasets present a formidable obstacle to accurately estimating this matrix. While imputation methods offer one avenue for addressing this challenge, they often entail a trade-off between computational efficiency and estimation accuracy. Consequently, attention has shifted towards direct parameter estimation, given its precision and reduced computational burden. In this paper, we propose Direct Parameter Estimation for Randomly Missing Data with Categorical Features (DPERC), an efficient approach for direct parameter estimation tailored to mixed data that contains missing values within continuous features. Our method exploits information from categorical features to improve covariance estimation for continuous variables, effectively leveraging the structure of mixed data. Through comprehensive evaluations, we show that DPERC is not only a valuable tool for visualizing correlation heatmaps but also performs competitively against the state-of-the-art techniques by improving up to 23.47% of the sum of squared errors.