Traditional tea classification methods relying on manual sensory and chemical analysis are time-consuming and inefficient. In contrast, computer vision-based tea classification offers a non-destructive and scalable alternative. However, existing approaches have limitations for real-world application, as their models are trained on limited laboratory data with insufficient diversity. To advance the state of computer-aided tea classification, this study makes two key contributions. First, we construct the Fine-Grained Tea (FGT) dataset, encompassing 35 fine-grained classes from the complete six Chinese tea categories, including a substantial number of real-world tea images. Second, we propose the Texture Enhanced Branching Network (TEBN), a novel two-stage classification model designed for fine-grained tea recognition. The first stage performs coarse classification, identifying images as either error-prone or regular subsets. Only error-prone images then pass through a second-stage Texture Enhanced Classifier (TEC) branch, which discriminates similar classes by capturing nuanced visual and semantic features. Experiments demonstrate the TEBN model’s effectiveness, achieving 89.35% top-1 accuracy on the FGT dataset - outperforming existing approaches. This work provides insights to guide future research in computer vision-based fine-grained tea classification for real-world applications.