The covariance matrix is a foundation in numerous statistical and machine learning applications, such as Principal Component Analysis, Correlation Heatmap, etc. However, missing values within datasets present a formidable obstacle to accurately estimating this matrix. While imputation methods offer one avenue for addressing this challenge, they often entail a trade-off between computational efficiency and estimation accuracy. Consequently, attention has shifted towards direct parameter estimation, given its precision and reduced computational burden. In this paper, we propose Direct Parameter Estimation for Randomly Missing Data with Categorical Features (DPERC), an efficient approach for direct parameter estimation tailored to mixed data that contains missing values within continuous features. Our method exploits information from categorical features to improve covariance estimation for continuous variables, effectively leveraging the structure of mixed data. Through comprehensive evaluations, we show that DPERC is not only a valuable tool for visualizing correlation heatmaps but also performs competitively against the state-of-the-art techniques by improving up to 23.47% of the sum of squared errors.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DPERC: Direct Parameter Estimation for Mixed Data with Random Missingness

  • Tuan L. Vo,
  • Uyen Dang,
  • Thu Nguyen,
  • Pål Halvorsen,
  • Michael A. Riegler,
  • Binh T. Nguyen

摘要

The covariance matrix is a foundation in numerous statistical and machine learning applications, such as Principal Component Analysis, Correlation Heatmap, etc. However, missing values within datasets present a formidable obstacle to accurately estimating this matrix. While imputation methods offer one avenue for addressing this challenge, they often entail a trade-off between computational efficiency and estimation accuracy. Consequently, attention has shifted towards direct parameter estimation, given its precision and reduced computational burden. In this paper, we propose Direct Parameter Estimation for Randomly Missing Data with Categorical Features (DPERC), an efficient approach for direct parameter estimation tailored to mixed data that contains missing values within continuous features. Our method exploits information from categorical features to improve covariance estimation for continuous variables, effectively leveraging the structure of mixed data. Through comprehensive evaluations, we show that DPERC is not only a valuable tool for visualizing correlation heatmaps but also performs competitively against the state-of-the-art techniques by improving up to 23.47% of the sum of squared errors.