FSOAR: Open-Set Recognition of Abnormal Action Under Few-Shot Learning
摘要
This paper primarily explores a methodology for the open-set recognition of abnormal action, particularly under the constraints imposed by few-shot learning. It examines the problem from dual perspectives: Few-Shot Learning (FSL) and Open-Set Recognition (OSR). The overarching objective of this research is to train a high-performance model capable of accurately identifying instances that do not conform to known classifications, especially in contexts where labeled abnormal action samples are notably scarce. FSL typically operates within closed-set paradigms, which inherently limits the model’s generalizability. In contrast, OSR necessitates extensive training datasets to bolster the model’ s generalization capability and robustness. Existing methods for abnormal action recognition often perform inadequately in open-set contexts with small sample learning, as they predominantly address either aspect in isolation, neglecting their potential synergistic integration. To address this shortfall, we introduce a approach for open-set recognition of abnormal action under few-shot, termed FSOAR (Few-Shot Open-Set Abnormal action Recognition), which adeptly amalgamates the Transformer with advanced contrastive learning techniques. By employing Transformer, we construct prototypes tailored to the query set, facilitating the measurement of similarity between these prototypes and the query set. Additionally, we devise a feature transformer based on both coarse-grained and fine-grained contrastive learning paradigms which improves sample discriminability within a feature space, thereby overcoming the limitations of previous methodologies. Comprehensive experimental results substantiate that our proposed FSOAR method achieves notable improvements over contemporary action recognition models across various public datasets.