Fine-Grained Visual Classification (FGVC) aims to distinguish images with subtle differences and limited inter-class variation. This study addresses the lack of understanding of dataset intricacies by analyzing the CUB-200-2011 (CUB) dataset. We identify four ultra-fine-grained subsets that significantly impact accuracy. While local visual cues can often distinguish samples, some fine-grained cases require global context for accurate classification. To address this, we propose a novel LSTM Decision Module (LSTM-DM) that combines local and global information flexibly. Additionally, a Region Selection Module (RSM) selects discriminative regions of fine-grained samples. These modules are integrated into the ViT architecture, enhancing its performance on FGVC tasks. A contrastive loss further improves feature representation by increasing the distance between confusing classes. Extensive evaluations show that our Gate-ViT outperforms existing state-of-the-art methods on four benchmark datasets.