With the vigorous development of video applications and the comprehensive upgrade of encryption protocols, video content review is becoming increasingly difficult. As a practical solution, video traffic identification has attracted more and more attention in recent years. In order to achieve more refined video traffic recognition, a primary problem to be solved is that the extracted traffic features lack higher-level semantic information to reflect changes in video content. Fortunately, under the same network conditions, similar types of video clips or traffic data generated by live video often have the same traffic change pattern, which allows different types of video traffic to be distinguished. Therefore, this paper extracts burst sequences from video traffic to represent video changes and constructs a graph structure based on burst sequences to extract high-level semantic information. With the powerful performance of a graph neural network, we transform video traffic classification into a graph classification problem and introduce GraphBurst, a model based on burst sequences and graph neural networks. We collected real-world traffic data from different network environments with five video types, including on-demand live streaming, with more than one hundred thousand streams. We launch a set of comparing experiments on the dataset, and compared with the existing methods, the experimental results show that our method has higher accuracy and interpretability.