Advances in Explainable Big Data Analytics for Enhanced Cybersecurity
摘要
This study presents an Intrusion Detection System (IDS) that addresses challenges in feature selection, model interpretability, and scalability for cybersecurity. By employing a two-step feature selection approach combining a correlation-based filter and Extra Trees feature selection, the proposed framework effectively reduces redundancy and noise while enhancing computational efficiency. Existing explainable IDSs typically rely on a single post-hoc interpreter and therefore fail to capture both global model structure and instance-specific root causes. To overcome the interpretability challenges of traditional machine learning models in IDS, the study proposes an IDS that couples a two-step feature-selection pipeline with a truly hybrid Explainable Artificial Intelligence (XAI) framework that fuses SHAPASH, ELI5, and compact Decision Tree Surrogate Model (DTSM). This framework combines local and global explainability, enabling both instance-level insights and broader feature interaction patterns using if-else rules. We analyze the performance of state-of-the-art machine learning methods on two cybersecurity datasets, UNSW-NB15 and BETH. Across UNSW-NB15 and BETH, Random Forest achieves 98% and 99% accuracy, respectively. Explanations materialize as human-readable if–else rules that can be deployed directly to firewall policies. This study contributes to advancing XAI in cybersecurity by providing a transparent, scalable, and effective IDS suitable for real-world operations.