The study of adversarial examples in deep neural networks has attracted great attention. Numerous methods improve adversarial robustness via shrinking the gap of features between natural examples and adversarial examples. Nevertheless, the role of individual features in adversarial robustness has not been explored adequately. In this paper, we delve into this problem from the perspective of spectral analysis in feature space. We find that while standardly trained deep models have features distributed dominantly along eigenvectors with large eigenvalues, eigenvectors with smaller eigenvalues are more sensitive to adversarial attacks. We attribute this phenomenon to the dominance of the top eigenvalues, linked to the concept of intrinsic dimensionality. The extracted features possess a small intrinsic dimensionality, enhancing generalization but resulting in the model overlooking diverse features. We propose a method called Feature Spectral Regularization (FSR) to penalize the largest eigenvalue, so as to spread the distribution of eigenvalues. Comprehensive experiments demonstrate that FSR is effective to alleviate the dominance of larger eigenvalues, increase the intrinsic dimensionality, and improve adversarial robustness on multiple datasets.