Surface defect detection of metallurgical products has become a focus of widespread concern in the metallurgical industry. Currently, the detection and determination of surface defects are predominantly performed by skilled inspectors, which is time-consuming and labor-intensive. Moreover, the performance of manual identification is constrained by the knowledge level and expertise of inspectors, making it difficult to accumulate, inherit and reuse the experience and knowledge contained in historical cases during long-term production processes. In this paper, we conduct in-depth mining of multi-source heterogeneous industrial data such as structured data, image data and text data in the steel industry. Through multimodal fusion methods such as aligning, disambiguating and merging synonymous entities or relationships, and reducing redundant information in the knowledge graph, we construct a multimodal knowledge graph. Finally, we combine graph neural networks to achieve knowledge reasoning for identifying different surface defect types of metallurgical products and tracing the causes of defects. In addition, we compare the performance of GCN, GAT and GraphSAGE on different datasets. The results show that GAT has relatively excellent performance.