Large-scale attacks targeting numerous Internet of Things (IoT) devices are significant threats. When such attacks occur, analysts at Security Operation Centers must manually analyze alerts from numerous IoT devices and create a large number of similar reports. If the SOC analysts can automatically evaluate whether the initially created report is reusable for other IoT devices (reusability evaluation), it will improve the efficiency of the alert analysis and the report creation. However, conventional methods applicable for the reusability evaluation suffer low accuracy and a lack of training data. Therefore, we propose a Reusability Evaluation Method (REM) that combines Large Language Models (LLMs) with Jaccard similarity to achieve high accuracy with a small amount of training data. REM uses Sentence ALBERT, which is one of the LLMs, and first fine-tunes it with a small amount of training data such as four pairs of a report and an alert set. Second, REM assigns weights to alerts on the basis of their relevance to the initially created report with Sentence ALBERT. Then, REM calculates the weighted Jaccard similarity between the alert set forming the basis of the report and a target alert set. If the weighted Jaccard similarity exceeds a predefined threshold, the SOC analysts reuse the report for the target alert set. We evaluate REM by using alert sets collected in an experimental environment simulating IoT devices and manually created reports. The results show that REM achieves a true positive rate of 82.63%, which is over 19% higher than that of conventional methods, at a false positive rate of 3.01%, even with a small amount of training data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Reusability Evaluation of Reports in Security Operation Centers for IoT with Sentence ALBERT and Jaccard Similarity

  • Masaru Matsubayashi,
  • Toshiki Shibahara,
  • Takuma Koyama,
  • Masashi Tanaka

摘要

Large-scale attacks targeting numerous Internet of Things (IoT) devices are significant threats. When such attacks occur, analysts at Security Operation Centers must manually analyze alerts from numerous IoT devices and create a large number of similar reports. If the SOC analysts can automatically evaluate whether the initially created report is reusable for other IoT devices (reusability evaluation), it will improve the efficiency of the alert analysis and the report creation. However, conventional methods applicable for the reusability evaluation suffer low accuracy and a lack of training data. Therefore, we propose a Reusability Evaluation Method (REM) that combines Large Language Models (LLMs) with Jaccard similarity to achieve high accuracy with a small amount of training data. REM uses Sentence ALBERT, which is one of the LLMs, and first fine-tunes it with a small amount of training data such as four pairs of a report and an alert set. Second, REM assigns weights to alerts on the basis of their relevance to the initially created report with Sentence ALBERT. Then, REM calculates the weighted Jaccard similarity between the alert set forming the basis of the report and a target alert set. If the weighted Jaccard similarity exceeds a predefined threshold, the SOC analysts reuse the report for the target alert set. We evaluate REM by using alert sets collected in an experimental environment simulating IoT devices and manually created reports. The results show that REM achieves a true positive rate of 82.63%, which is over 19% higher than that of conventional methods, at a false positive rate of 3.01%, even with a small amount of training data.