Missing data has been a significant challenge in data analysis, reducing the reliability of predictions and disrupting data patterns. In dengue-related datasets, incomplete data has complicated outbreak prediction and public health responses. Traditional imputation techniques often produce biased results, while advanced techniques are computationally intensive and lack transparency. To address this, a novel ensemble weighted average imputation technique combining XGBoost and MiceForest named Gradient-Forest Weighted Average Imputer (GF-WAI) has been proposed. Six additional imputation techniques have been implemented for comparison, and predictive accuracy has been evaluated using RFC, SVC, and NBC. Evaluations have been conducted on both dengue datasets of 1,003 and 10,000 records. The proposed method has outperformed others, achieving an MAE of 641.26 and RMSE of 6743.63 on the small dataset, with significant improvements on the larger dataset, where an MAE of 144.49 and RMSE of 2408.32 have been achieved. An accuracy of 99.90% and an F1 score of 99.89% have been achieved by GF-WAI with RFC on the larger dataset, while 98.26% accuracy and an F1 score of 98.25% have been achieved on the smaller dataset. LIME, an explainable AI tool, has been used to further evaluate GF-WAI, providing insights into feature contributions. These findings have highlighted the effectiveness of the proposed technique in addressing missing data challenges, supporting better public health decision-making for dengue intervention strategies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Improving Missing Data Imputation with GF-WAI: An Explainable AI-Based Ensemble Method for Dengue Disease Prediction

  • Sigma Khanam,
  • Maisha Maliat,
  • Bineta Abedeen,
  • Farjana Khan,
  • Farzana Tasnim,
  • Shefayatuj Johara Chowdhury,
  • Tanjim Mahmud,
  • Kaushik Deb,
  • Mohammad Shahadat Hossain,
  • Abubokor Hanip

摘要

Missing data has been a significant challenge in data analysis, reducing the reliability of predictions and disrupting data patterns. In dengue-related datasets, incomplete data has complicated outbreak prediction and public health responses. Traditional imputation techniques often produce biased results, while advanced techniques are computationally intensive and lack transparency. To address this, a novel ensemble weighted average imputation technique combining XGBoost and MiceForest named Gradient-Forest Weighted Average Imputer (GF-WAI) has been proposed. Six additional imputation techniques have been implemented for comparison, and predictive accuracy has been evaluated using RFC, SVC, and NBC. Evaluations have been conducted on both dengue datasets of 1,003 and 10,000 records. The proposed method has outperformed others, achieving an MAE of 641.26 and RMSE of 6743.63 on the small dataset, with significant improvements on the larger dataset, where an MAE of 144.49 and RMSE of 2408.32 have been achieved. An accuracy of 99.90% and an F1 score of 99.89% have been achieved by GF-WAI with RFC on the larger dataset, while 98.26% accuracy and an F1 score of 98.25% have been achieved on the smaller dataset. LIME, an explainable AI tool, has been used to further evaluate GF-WAI, providing insights into feature contributions. These findings have highlighted the effectiveness of the proposed technique in addressing missing data challenges, supporting better public health decision-making for dengue intervention strategies.