Revolutionising anomaly detection: a hybrid framework for anomaly detection integrating isolation forest, autoencoder, and Conv. LSTM
摘要
Anomaly detection is a critical task in various domains, such as cybersecurity, healthcare, and finance, where identifying rare and irregular patterns is essential. Traditional methods often struggle with imbalanced data, high dimensionality, and temporal dependencies, limiting their applicability in real-world scenarios. This study aims to develop a robust and scalable hybrid anomaly detection framework that effectively handles the complexities of multidomain data and improves detection accuracy and efficiency. We propose a novel hybrid model that integrates three complementary components: Isolation Forest for efficient outlier partitioning, autoencoder for latent feature extraction and reconstruction error analysis, and ConvLSTM for capturing spatial–temporal patterns in sequential data. The model was evaluated on diverse datasets, including synthetic Gaussian data, KDD Cup 1999 (network intrusion), credit card fraud detection, and breast cancer (healthcare), along with additional IoT and streaming datasets. A comprehensive preprocessing pipeline and hyperparameter optimisation strategy were employed to enhance performance. The proposed model achieved up to 99.5% accuracy and 98.2% F1-score, outperforming the baseline and state-of-the-art hybrid methods in terms of both predictive performance and execution time. Cross-domain validation confirmed strong generalisation capabilities without retraining. The hybrid framework demonstrates superior adaptability, interpretability, and scalability, making it a promising solution for real-time anomaly detection in complex and high-dimensional dynamic environments.