Application of Ensemble Learning Based on High-Dimensional Features in Financial Big Data
摘要
Currently, most of the research on Internet financial models is based on traditional machine learning models, which often fail to adequately capture the complex features and potential non-linear interactions within financial data. In addition, the training process of the models is usually complex and time-consuming, making it difficult to meet the demand for rapid processing and analysis of financial data. To address these issues, this paper proposes a new approach that combines high-dimensional feature engineering with ensemble learning. Unlike most existing feature engineering methods, this paper specifically focuses on the high-dimensional features of financial data, and thoroughly investigates how to efficiently construct and select more useful features to fully explore the potential information of the data. We combine tree-based ensemble methods (e.g., LightGBM and XGBoost) with neural network models to construct a multi-level ensemble learning framework for complex financial data features. Experimental results show that the method not only generates high-quality and informative features, but also significantly enhances the robustness and prediction accuracy of the model, effectively solving the challenges for modeling and training complex financial data, especially when facing large-scale and multi-dimensional datasets.