NLCC: Noisy Label Correction with CLIP for Robust Few-Shot Learning
摘要
In social media crowdsourced annotation, image-text multimodal data grows rapidly, but user-generated labels often contain noise (e.g., mislabeling a wolf as a husky). This issue is particularly prominent during the cold start phase of new users, where few-shot data with noisy labels exerts a more significant influence due to the limited data for robust learning. To address this, we propose a multimodal approach for correcting noisy labels. First, we leverage the image-text matching capability of the pre-trained CLIP model to filter mismatched multimodal data and correct erroneous labels via similarity metrics. However, since CLIP’s correction results are not entirely accurate, we introduce a multi-label mechanism that incorporates the top-k labels filtered by CLIP with weighted integration into training, forming a plug-and-play multimodal noisy label correction method. This provides a solution to the cold start problem with noisy behaviors. We embedded our method in 11 datasets and two classic methods, and experimental results validate its effectiveness.