Robust machine learning and ensemble learning approach to predict variation in experimental data for multiple measurements and anomalies
摘要
In the field of chemistry, machine learning is widely used to develop desired properties and activities y of compounds and products from experimental conditions x. As y is measured several times, a mathematical model is constructed with y as the average value of these measurements, which cannot evaluate the variability of y. Therefore, a method was proposed to predict the variability of y by creating N sub-datasets with selected y values for each sample and constructing multiple models. However, this method reduces prediction accuracy when there are abnormal values due to measurement errors. To address these issues, we proposed a robust method that constructs multiple sub-datasets and selects only the models with the lowest mean absolute error for predictions. Validation on a film thickness and haze (light scattering intensity) dataset showed that the proposed method outperformed conventional approaches, including those that remove anomalies in advance, in predicting both the mean and variation of y. The proposed method could improve accuracy in datasets with multiple values of y and containing abnormal values without removing samples.
Graphical abstract