Data Pre-processing Techniques
摘要
The pre-processing of data is the most important initial step in the analysis of spectroscopic information, and this activity is widely performed in various scientific and industrial applications. These pre-processing techniques are known to enhance the quality of the raw spectral data by reducing noise, correcting instrumental and environmental variations, as well as to facilitating accurate and reliable interpretation of spectral information for real life application. For fluorescence spectroscopy, the key pre-processing methods commonly employed in data analysis can be categorized into: normalization, peak detection, noise reduction, resolution enhancement, and baseline correction. The impact of these pre-processing techniques in each category on the performance of subsequent multivariate methods is worthwhile. The strength of each pre-processing underscores the importance of selecting an appropriate one that is tailored to the characteristics of the spectral data, the outcome required and the objectives of the analysis to ensure the reliability, robustness and meaningful results. By improving the quality and interpretability of spectral data, pre-processing techniques play a pivotal role in enhancing the accuracy and reliability of spectroscopic analyses across diverse applications. Among the different types available, standardization, Multiplicative Scatter Correction (MSC) and Standard Normal Variant (SNV) were found to be effective while the derivative types are not favorable. However, the best pre-processing strategy could involve the combining of several techniques, for instance, applying baseline correction followed by smoothing and normalization. More so, it is crucial to validate the pre-processing techniques by assessing their impact on the subsequent analysis, such as either classification or quantification challenges being studied in the food industry.