KL-CLIP: a K-means learning model for zero-shot anomaly segmentation
摘要
Recently, large-scale visual-language models, such as CLIP, have demonstrated significant potential in zero-shot anomaly segmentation (ZSAS) tasks. These models can detect anomalies in unseen products by integrating image and text features. However, in practical scenarios like strip steel with highly repetitive textures, existing methods primarily emphasize global feature matching. They often struggle to effectively capture small or subtle anomalies and face challenges as textual cues become confused with defect features, particularly given the diverse defect categories on strip steel surfaces. To address these challenges, we propose a K-means Learning model (KL-CLIP) for ZSAS task based on CLIP. KL-CLIP incorporates feature weighting into CLIP and optimizes the model using an activation function, attention mechanisms, and modified loss functions. We propose the KM module, which adjusts CLIP by clustering features, generating weighting factors, and effectively addressing the diverse defect categories on strip steel surfaces. Extensive experiments on the strip steel defect dataset (FSSD-12) and other industrial defect datasets demonstrate that KL-CLIP outperforms existing ZSAS methods and generalizes better across various industrial applications. On the FSSD-12 dataset, KL-CLIP achieves a 1.5% AP improvement, a 1% PRO improvement, and a 0.9% pixel-level AUROC improvement over the baseline.