Recent years have witnessed substantial advancements in industrial anomaly detection for visual localization and reconstruction, with mainstream methodologies primarily focusing on reconstruction-based and embedding-based approaches. However, current techniques predominantly generate pixel-level diagnostic maps and anomaly-free reconstructed images that lack semantic characterization of defective regions, necessitating secondary manual verification and consequently limiting practical applicability. Furthermore, mask maps produced by existing models typically suffer from noise interference, edge blurring, and regional discontinuities, which compromise detection accuracy and undermine the reliability of semantic analysis. To address these challenges, this paper proposes the SEM-IDET framework, an end-to-end solution that integrates MLLMs for generating semantic anomaly descriptions with training-free mask refinement to enhance semantic interpretation accuracy. The proposed method employs median filtering, morphological operations, and adaptive threshold processing to significantly improve mask localization precision and spatial continuity. The refined masks are jointly processed with original images through MLLMs, enabling the model to generate precise semantic descriptions of anomalous regions. This integration substantially improves the interpretability and practical utility of diagnostic outputs. Experimental results demonstrate that SEM-IDET outperforms existing state-of-the-art models while exhibiting superior semantic matching capabilities.