Evaluating Algorithms for Missing Value Imputation in Real Battery Data
摘要
With the growing number of data-driven services, effective methodologies to identify errors and rectify missing values within data are required to ensure data quality. This paper examines the performance of three supervised prediction and two imputation techniques for missing value imputation on real, structured data from high-voltage batteries in the automotive industry. The cost of incorrect data can be very high in this application. In this study, starting with an error-free data set, missing values are generated according to the missing-at-random mechanism. Thereafter, the decision tree, multilayer perceptron, k-nearest neighbour, and support vector machine algorithms are evaluated for classification and regression tasks. Further, the k-nearest neighbour imputer and the multiple imputation by chained equations algorithm are evaluated as imputation techniques. The performance of these algorithms is compared based on their reliability and error metrics for three categorical and seven continuous features. The paper shows that features, where the standard deviation is less than the mean, are predicted more reliably than features with larger spreads. It is seen that more complex algorithms, i.e. support vector machine and multi-layer perceptron, perform better for more complex features, i.e. features with a higher cardinality and a wide range. The k-nearest neighbour algorithm emerges as the best performing, demonstrating efficient learning and generalisation across all features. The approach used in this research identifies the strengths and weaknesses of various machine learning algorithms in handling imputation tasks. This research informs the selection and combination of methods for better error correction and data management, ultimately supporting more reliable and efficient operations. This paper has shown that machine learning algorithms can be used for missing value imputation and error correction, with k-nearest neighbour and multiple imputation by chained equations providing efficient and reliable estimates for missing values, with minimal training effort.