Testing Effects of Potential Adversary Manipulations of Ship-Tracking Data
摘要
This work addressed ways of making machine learning more robust against adversaries trying to manipulate its input data, as with input sensor data an adversary can at least partially control. The approach we explored was training and comparing alternative models for the same machine-learning data. Our project tested this on real ship-tracking data from the public AIS (Automated Identification System) database available from the U.S. Coast Guard for U.S. coastal waters. After filtering out stationary ships and tracking errors, we built ship tracks and tested to identify the ten most important features of tracks that help classify ship types, with the goal of identifying the types of ships not reporting or falsely reporting their identities. We then systematically perturbed the data to various degrees to see how that affected the ship-type classifications with eight standard machine-learning methods. We were particularly interested in nonlinear effects that did not uniformly decrease classification accuracy with the degree of perturbation. Such effects could be exploited by adversaries trying to deceive with their ships. Our results did find some interesting weaknesses in the methods, and our methodology is general enough to be applied to other kinds of tracking data.