Sheep behaviour can reflect their growth and health. However, the current sheep behaviour recognition studies have yet to focus on statistics on the duration of various behaviours for individual sheep. Traditional behavioural recognition methods can overlook abnormal sheep behaviour such as prolonged lying or not eating. Therefore, we propose an advanced framework for statistically analyzing the duration of sheep behaviours within a farm environment. This paper constructed a dataset of sheep behaviour images collected from a natural farm environment, including walking, standing, eating, lame, lying, licking, and attacking. Based on the Vision Transformer (ViT) method and the YOLOv8 model, a sheep tracking model, ViTSORT, and a sheep behaviour recognition model, YOLOv8-MS, are presented for the duration of sheep behaviours. The experimental results show that ViTSORT can solve the problem of tracking target loss when sheep cover each other. Meanwhile, the YOLOv8-MS model achieves a precision of 96.9% mAP on our sheep behaviour dataset, and the detection speed is 196 FPS, higher than previous methods.