In the farming industry, to cope with problems such as complex pasture environments and dense targets, which lead to increased difficulty in recognising and thus quickly classifying and automatically identifying cattle breeds to improve accuracy. In this paper, an RTDETR-Refa (RepConv Efficient Faster Attention) algorithm based on ResNet18 backbone network is proposed for cattle breed classification and identification. First, new improvements are made to the ResNet18 backbone network: the Faster-Block module is introduced to improve the feature extraction network and increase the computational speed without sacrificing the accuracy; the 1 \(\times\) 1 convolution in the Faster-Block module is replaced by a 3 \(\times\) 3 convolution using the RepConv reparameterised with the RepVGG block, which makes the algorithm more lightweight and improves the inference speed. Second, in order to enhance the feature transformation and classification, the Efficient Multiscale Attention (EMA) module is added after the Faster-Block module at different stages. Finally, the above improved Faster-Block module is used to replace the 4-layer BasicBlock after the 3 convolutional layers in the backbone network of ResNet18. The training test results of the RTDETR-Refa algorithm are compared with other classical models to validate the superiority of the RTDETR-Refa algorithm. The average accuracy of the RTDETR-Refa algorithm on the bovine The average accuracy of RTDETR-Refa algorithm on the classification training set is 91.6%, which is 0.8% higher than the original model and 0.9–5.2% higher than other classical models. The experimental results show that the RTDETR-Refa model proposed in this paper is able to identify and classify different breeds of cattle while guaranteeing similar detection speed, which proves the feasibility of convolutional neural networks in breed identification and classification.