Due to the characteristics of low latency, high dynamic range and high time resolution, event cameras could be used as a powerful supplement to conventional cameras. In recent years, though object tracking algorithms based on frame-event fusion have made great progress in terms of accuracy, most of them couldn’t deal with the huge frequency gap between these two modalities in an ideal way, failing to take advantage of the high-speed nature of the event camera. In this paper, we propose a novel framework for real-time online tracking, which utilizes two networks with different inference speeds and estimates the target state with both of them, realizing high-frequency tracking while maintaining high accuracy.