Robust Distributed Bandit with Malicious Agents
摘要
During recent years, distributed bandit learning has gained increasing popularity in both academics and industry. In a typical setting, a central server coordinates a group of agents to interact with a bandit model and minimize its cumulative regret. However, existing works assume an ideal scenario where each agent always reports its true data to the server, which is unrealistic in the real world. Therefore, we investigate a centralized multi-agent bandit model with the existence of malicious agents, who may report arbitrary data and mislead the server during the whole learning process. Moreover, we consider more general settings to include the homogeneous and heterogeneous rewards for different agents. In the homogeneous setting, we adopt the trimmed method to filter the malicious data and design a TUCB (Trimmed Upper Confidence Bound) algorithm to conduct the online process. For the heterogeneous setting, we utilize an enhanced \(\beta \) -trimmed method and propose the \(\beta \) -TUCB algorithm accordingly. The theoretical analysis reveals that our algorithms can achieve significant sublinear regret for both settings—i.e., our methods can make robust reward estimations and minimize the cumulative regret even with the existence of malicious agents. Extensive simulations are conducted to demonstrate the superiority of our algorithms.