NetDetective: Dynamic Cyber Situational Awareness
摘要
Cyber situational awareness (CSA) is crucial for intrusion detection and response. Traditionally, CSA relied on human experts who have intimate knowledge of their defended networks and typically assumed static behaviours over the observed period of time. Effective CSA requires tools and techniques to perceive, comprehend, and predict the dynamic behaviours of defended networks. We investigate in this work the potential of machine-learning models for CSA—in particular, classifying dynamic network host behaviours from traffic data. We annotated an existing public dataset to create the first publicly-available dataset for host-behaviour characterisation with per-hour labels to better capture the dynamic nature of host behaviour. We then developed NetDetective, a machine-learning algorithm that achieved a high level of performance on our dataset, with a 5-fold cross-validated macro-averaged \(F_1\) score of 0.958 on our static labels and 0.929 on our dynamic labels. This algorithm used a gradient-boosted decision tree model trained on a set of features including network flow statistics, and port usage at both a coarse and fine-grained level.