Traditional k-Nearest Neighbors (KNN) classification assumes equal feature importance, which limits its effectiveness in domains where features exhibit inherently different levels of relevance. This study introduces a systematic feature importance weighted distance framework that integrates Random Forest derived importance scores into classical distance metrics. Ten distance measures, namely Euclidean, Manhattan, Cosine, Lorentzian, Canberra, Squared Chord, Chi Square, Whittaker’s Index of Association Distance, Motyka, and Hassanat, are adapted by embedding normalized feature importance weights, allowing informative features to contribute more strongly to similarity computation while reducing the influence of less relevant ones. Comprehensive evaluation across seven publicly available datasets, including five medical and two non-medical datasets, demonstrates consistent performance improvements. Using stratified 5 fold cross validation repeated over ten random seeds, the proposed weighted variants achieve an average accuracy gain of approximately 5.01 percentage points across all metrics, datasets, and values of k. Performance is improved or maintained in 91.1% of all tested configurations spanning five values of k (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">$k \in \{1,3,5,7,9\}$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>k</mi> <mo>∈</mo> <mo stretchy="false">{</mo> <mn>1</mn> <mo>,</mo> <mn>3</mn> <mo>,</mo> <mn>5</mn> <mo>,</mo> <mn>7</mn> <mo>,</mo> <mn>9</mn> <mo stretchy="false">}</mo> </mrow> </math></EquationSource> </InlineEquation>). Wilcoxon signed rank tests confirm statistical significance (<InlineEquation ID="IEq2"> <EquationSource Format="TEX">$p < 0.05$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>p</mi> <mo><</mo> <mn>0.05</mn> </mrow> </math></EquationSource> </InlineEquation>) in 78.0% of configurations, while sensitivity analysis indicates stable behavior across different Random Forest hyperparameter settings. Overall, the proposed framework provides an interpretable and effective mechanism for incorporating feature relevance into similarity based classification.

Feature-Importance-Weighted Distance Metrics for Enhanced k-Nearest Neighbors Classification in Medical Diagnosis

期刊论文
出版模式：

开放获取
发表日期：

2026年4月9日

查看全文

Mohammed Addou,
El Bekkaye Mermri,
Mohammed Gabli

摘要

Traditional k-Nearest Neighbors (KNN) classification assumes equal feature importance, which limits its effectiveness in domains where features exhibit inherently different levels of relevance. This study introduces a systematic feature importance weighted distance framework that integrates Random Forest derived importance scores into classical distance metrics. Ten distance measures, namely Euclidean, Manhattan, Cosine, Lorentzian, Canberra, Squared Chord, Chi Square, Whittaker’s Index of Association Distance, Motyka, and Hassanat, are adapted by embedding normalized feature importance weights, allowing informative features to contribute more strongly to similarity computation while reducing the influence of less relevant ones. Comprehensive evaluation across seven publicly available datasets, including five medical and two non-medical datasets, demonstrates consistent performance improvements. Using stratified 5 fold cross validation repeated over ten random seeds, the proposed weighted variants achieve an average accuracy gain of approximately 5.01 percentage points across all metrics, datasets, and values of k. Performance is improved or maintained in 91.1% of all tested configurations spanning five values of k ( $k \in \{1,3,5,7,9\}$ $k \in {1, 3, 5, 7, 9}$ ). Wilcoxon signed rank tests confirm statistical significance ( $p < 0.05$ $p < 0.05$ ) in 78.0% of configurations, while sensitivity analysis indicates stable behavior across different Random Forest hyperparameter settings. Overall, the proposed framework provides an interpretable and effective mechanism for incorporating feature relevance into similarity based classification.