During the production of solar wafers on automated production lines, wafers frequently break. To meet silicon wafer manufacturers’ demands for high precision, reliability, and real-time capabilities in online defect detection, this chapter proposes a deep neural network for silicon wafer defect detection based on prior knowledge. Initially, following the prior knowledge that silicon wafer damage often occurs at the edges, an additional prior mask branch is added to the YOLOv5 model, making the network more focused on the edges of the wafers. Secondly, we introduce a parameter-free attention module, SimAm, into YOLOv5, which enhances the model’s ability to extract features of defects on silicon wafers without increasing the model’s complexity. Compared to YOLOv5s, the experimental results show that the performance of our network improved by 2.1%, meeting the need for online defect detection of silicon wafers.