Scholarlink

<p>Effective log assessment is a cornerstone for ensuring the reliability and smooth functioning of large-scale, cloud-based infrastructures. As these environments continuously grow in scope and intricacy, traditional monitoring approaches often struggle to keep up, leading to unnecessary computational strain and diminished system performance. To address these challenges, this research introduces a hybrid framework that integrates natural language processing (NLP) with an optimized LightGBM classifier, tailored for anomaly detection in cloud-generated system logs. Central to the design is a thorough preprocessing pipeline that filters irrelevant information and minimizes bias, thereby sharpening anomaly detection accuracy. The experimental setup explored multiple NLP-based preprocessing strategies, including TF-IDF, BERT, and Word2Vec implementations (employing spaCy and Gensim). Classification relied on the LightGBM model, whose hyperparameters were refined using a customized particle swarm optimization (PSO) metaheuristic. This modified optimization routine boosted both predictive accuracy and model robustness. Results from the study reveal that the combined system significantly enhanced both the identification and classification of anomalies in cloud logs. The most effective configuration achieved up to 100% accuracy under the evaluated experimental conditions, highlighting the framework’s potential to strengthen security within cloud ecosystems. To ensure statistical soundness, extensive comparative evaluations were conducted across all configurations. Additionally, model interpretability was improved through SHapley Additive exPlanations (SHAP), which provided transparent insights into the role of individual features in shaping classification outcomes.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Comparative evaluation of natural language processing approaches with particle swarm optimized LightGBM for anomaly detection in cloud system logs

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月13日

查看全文
  • Slavisa Djukanovic,
  • Sasa Stojanovic,
  • Bosko Nikolic,
  • Snezana Anetic,
  • Tamara Zivkovic,
  • Miodrag Zivkovic,
  • Vladimir Simic,
  • Nebojsa Bacanin

摘要

Effective log assessment is a cornerstone for ensuring the reliability and smooth functioning of large-scale, cloud-based infrastructures. As these environments continuously grow in scope and intricacy, traditional monitoring approaches often struggle to keep up, leading to unnecessary computational strain and diminished system performance. To address these challenges, this research introduces a hybrid framework that integrates natural language processing (NLP) with an optimized LightGBM classifier, tailored for anomaly detection in cloud-generated system logs. Central to the design is a thorough preprocessing pipeline that filters irrelevant information and minimizes bias, thereby sharpening anomaly detection accuracy. The experimental setup explored multiple NLP-based preprocessing strategies, including TF-IDF, BERT, and Word2Vec implementations (employing spaCy and Gensim). Classification relied on the LightGBM model, whose hyperparameters were refined using a customized particle swarm optimization (PSO) metaheuristic. This modified optimization routine boosted both predictive accuracy and model robustness. Results from the study reveal that the combined system significantly enhanced both the identification and classification of anomalies in cloud logs. The most effective configuration achieved up to 100% accuracy under the evaluated experimental conditions, highlighting the framework’s potential to strengthen security within cloud ecosystems. To ensure statistical soundness, extensive comparative evaluations were conducted across all configurations. Additionally, model interpretability was improved through SHapley Additive exPlanations (SHAP), which provided transparent insights into the role of individual features in shaping classification outcomes.