Scholarlink

<p>In the rapidly progressing field of smart home technology, safeguarding the security of interconnected devices and systems is critical. Unsupervised anomaly detection is a recognized solution; however, its effectiveness heavily depends on the hyperparameter optimization process. In this work, we hypothesize that metrics tailored for imbalanced data, such as the Matthews correlation coefficient (MCC), can guide the development of more robust and generalizable unsupervised anomaly detection models. To test this hypothesis, we systematically evaluate the impact of optimizing seven distinct metrics (e.g., MCC, BACC, AUC-PR) across four widely used smart home datasets (N-BaIoT, IoTID20, Bot-IoT, and ToN-IoT). Our results show that MCC optimized models achieve an average normalized MCC of 0.801, outperforming those optimized for accuracy (0.781) and AUC-ROC (0.733), and reducing the cumulative distance to ideal performance by 66% compared to conventional metric optimization. These findings underscore the importance of metric selection in unsupervised settings, offering a foundation for building more reliable and generalizable security models for smart home environments. To our knowledge, this is the first work to systematically assess the effects of optimizing distinct evaluation metrics across multiple smart home datasets.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Hyperparameter Optimization and Evaluation Metrics for Unsupervised Anomaly-Based Cyberattack Detection in Imbalanced Smart Home Datasets

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年9月12日

查看全文
  • Juan-Ignacio Iturbe-Araya,
  • Helena Rifà-Pous

摘要

In the rapidly progressing field of smart home technology, safeguarding the security of interconnected devices and systems is critical. Unsupervised anomaly detection is a recognized solution; however, its effectiveness heavily depends on the hyperparameter optimization process. In this work, we hypothesize that metrics tailored for imbalanced data, such as the Matthews correlation coefficient (MCC), can guide the development of more robust and generalizable unsupervised anomaly detection models. To test this hypothesis, we systematically evaluate the impact of optimizing seven distinct metrics (e.g., MCC, BACC, AUC-PR) across four widely used smart home datasets (N-BaIoT, IoTID20, Bot-IoT, and ToN-IoT). Our results show that MCC optimized models achieve an average normalized MCC of 0.801, outperforming those optimized for accuracy (0.781) and AUC-ROC (0.733), and reducing the cumulative distance to ideal performance by 66% compared to conventional metric optimization. These findings underscore the importance of metric selection in unsupervised settings, offering a foundation for building more reliable and generalizable security models for smart home environments. To our knowledge, this is the first work to systematically assess the effects of optimizing distinct evaluation metrics across multiple smart home datasets.