Adversarial training during the pre-training stage has been recognized as an effective approach to enhance the robustness of self-supervised learning (SSL) models. However, the combination of computationally intensive self-supervised pre-training and adversarial training results in a substantial increase in computational overhead. To address this challenge, this paper shifts focus to leveraging the inherent robustness of standard pre-trained models, rather than relying on adversarial pre-training. We first investigate the impact of fine-tuning strategies on the robustness of SSL models from the perspective of loss landscape flatness. Our findings indicate that full-parameter fine-tuning tends to compromise the robustness of pre-trained models, while classifier-only fine-tuning exhibits limited robustness transferability. Motivated by these observations, we propose Consistent LoRA (CLoRA), a novel fine-tuning method. We show that CLoRA enables effective transfer of the robustness from standard pre-trained models to downstream tasks without requiring adversarial pre-training. Experimental results on benchmark datasets demonstrate that CLoRA, based on standard pre-training, achieves significantly better adversarial robustness than state-of-the-art (SOTA) methods relying on adversarial pre-training, with minimal computational overhead.