Effective Pre-processing of Datasets by Removing Anomalies with OptiContamFinder on Isolation Forest for Stock Market forecasting and Nowcasting
摘要
Effective stock market forecasting and nowcasting requires effective data collection pre-processing. Traditional methods are often unable to identify and remove abnormalities, which be able to have a noteworthy impact on predictive model presentation. Current limitations of pre-processing stock market datasets require appropriate methods for correcting anomalies in an appropriate manner, resulting in low model performance In this case, researchers propose a different approach so detecting and extracting anomalies in stock market data sets efficiently using OptiContamFinder is proposed. Abnormalities, such as extreme changes can distort the underlying assumptions and correlations in the data, reducing the accuracy of the prediction model. To address these limitations, develop OptiContamFinder, which enables anomaly detection with Isolation Forest. This method excels in reducing anomalies by generating random forests of isolated trees and identifying remote areas with short path lengths. OptiContamFinder improves anomaly detection by striking a balance between anomaly removal and data preservation by providing optimization options for determining contamination thresholds. The entire process consists of gathering historical stock market data, cleaning and identifying important characteristics, normalizing or scaling the features, and using OptiContamFinder to discover and remove anomalies. After pre-processing, the predictive models are trained and tested against acceptable performance metrics. By implementing both nowcasting and forecasting for the pre-processing of stock market anomaly detection shows pre-processed accuracy of 85% and 88%. By successfully detecting and cancelling anomalies, OptiContamFinder prolongs forecasting models, resulting in better forecasts of stock price movements using python.