<p>Existing open-set detection methods often rely on deep networks trained on known categories, which can lead to overfitting and limit generalization to unseen anomalies. To address this, we propose a novel network architecture that combines local feature extraction with global feature modeling to enhance both feature extraction and defect detection. Although the Vision Transformer (ViT) has shown promise in capturing complex and variable anomaly patterns, it still suffers from challenges related to high computational cost and difficulties in handling local feature interactions effectively. This motivates us to design an efficient Adaptive Feature Fusion Convolution (AFFConv) module and boost feature fusion by employing KMeans clustering to select reference features, maximizing the utilization of multi-level feature information. Similarly, a multilayer perceptron (MLP) is integrated into the head module to strengthen the network’s ability to learn complex nonlinear features. Additionally, by adopting a dynamic head score fusion strategy, our model adaptively combines the outputs of multiple heads using learnable weights, thereby enhancing both detection flexibility and accuracy. Extensive experiments demonstrate that our VDA achieves the best performance on multiple open-set defect detection datasets, excelling in the identification of complex, unknown anomalies with high stability and accuracy.</p>