Representation space evaluation of feature extractors for model transferability in fine-grained recognition
摘要
This paper introduces an evaluation pipeline for transferability estimation in fine-grained visual recognition tasks, with a focus on hand gesture recognition. The proposed framework provides a resource-efficient and interpretable tool for validating model suitability to downstream tasks without exhaustive fine-tuning. By analyzing the representation spaces of both convolutional and transformer-based vision models through dimensionality reduction and clustering, our pipeline incorporates a similarity-aware scoring technique that explicitly considers both intra-class variability and inter-class similarity. In experiments, our framework consistently ranked the vision transformer with advanced pre-training highest, while assigning lower importance to the standard vision transformer and convolutional models. These results indicate that the proposed approach can effectively highlight models best suited for fine-grained recognition. We provide the source code and experimental setup at https://github.com/ADA-SITE-JML/gesture_face_features.