<p>Industrial defect detection scenarios necessitate the use of efficient inference networks. However, high-precision segmentation algorithms utilizing self-attention mechanisms often face challenges related to quadratic complexity and prolonged inference times. To address these issues, this manuscript proposes LDD-CAP, a lightweight defect detection network featuring a dual-branch architecture and incorporating class prior information. To achieve optimal detection performance while balancing model size and inference speed, this study introduces an efficient category prior network (ECPN) designed to serve as the feature extraction backbone for the segmentation algorithm, effectively capturing a comprehensive combination of spatial and channel features. Specifically, to enhance semantic feature representation while simplifying computation, a Split Depth Mixed External Attention (SDMEA) strategy is proposed. Additionally, to strengthen the attention pairwise affinity toward defective samples, a Supervised Attention Focusing (SAF) method is introduced, which imposes hard constraints on category-level feature distributions, thereby enhancing global information parsing while extracting local features. Finally, a Dual Branch Segmentation Architecture (DBSA) is employed to produce the segmentation output. Experiments on three defect detection datasets demonstrate that the proposed LDD-CAP network outperforms state-of-the-art models in both mean Intersection-over-Union (mIoU) and mean Accuracy (mAcc) (NEU-seg: 88.49%, Magnetic-Tile: 87.00%, FSSD-12: 91.12%), confirming its effectiveness. Additionally, its lightweight structural design facilitates the efficient deployment of the model on high-performance computing platforms in industrial settings, enabling real-time industrial defect detection across large volumes of data, thus enhancing production efficiency and product quality.</p>