Adversarial training and attacks expose the weak spot in deep neural networks, which generally produce wrong classifications when subjected to subtle, imperceptible perturbations. This vulnerability poses a significant uncertainty in the security of these networks. Most current research focuses on defenses and robust training methods for image processing, with relatively limited studies on defenses against adversarial attacks in natural language processing (NLP). In this study, we suggest an improved method for fine-tuning machine learning, named Trade-Off Defense (TOD), to address the overfitting issue on adversarial samples in existing NLP tasks. We optimize the fine-tuning methods of NLP models by designing a loss function based on the concept of alignment to balance the prediction value of adversarial samples and original samples, thereby improving the robustness and generalization ability of the model. We adopted BERT as the base model and, tested the effectiveness of TOD on public datasets including IMDB and Yelp. The experimental findings indicate that TOD significantly lowers the success rate of adversarial attacks between different training samples without affecting model accuracy and interpretability, consequently enhancing the robustness and generalization ability of the algorithm. In addition, TOD also demonstrates higher standard accuracy and good interpretability on the test datasets. TOD is open-source and accessible at this repository: https://github.com/jiangdaolime/TOD .