Preprocessing and Feature Selection Techniques for Enhancing AI Model Performance on Intrusion Detection System Datasets
摘要
Effective preprocessing and feature selection are pivotal for optimizing AI model performance in cybersecurity. This work focuses on the application of advanced preprocessing techniques to the CSECICIDS2017 and CICIDS2018 datasets, emphasizing the use of Naive Bayes and Random Forest algorithms for feature selection alongside Correlation-based Feature Selection (CFS). These methods identify the most relevant features, ensuring the refinement of data for subsequent analysis. Additionally, t-SNE (t-distributed Stochastic Neighbor Embedding) is employed for visualizing high-dimensional data, providing insights into feature distribution and model performance. These methodologies aim to streamline the preprocessing pipeline, improve feature relevance, and facilitate better understanding of data patterns, ultimately advancing the utility of machine learning models in cybersecurity.