Large pre-trained language models (PLMs) are being updated continuously. To handle massive amounts of training data and adapt to various downstream tasks, model fine-tuning has become increasingly crucial. Among the existing parameter-efficient fine-tuning methods, Low-Rank Adaptation (LoRA) and its variants are quite popular because they do not incur additional inference costs. However, the current LoRA-based methods still have room for improvement in terms of their ability to adapt to specific downstream tasks. Considering the differences in the importance of different layers in large models, we optimize the selection of rank values in LoRA, making LoRA pay more attention to the information in the deeper layers of the model through linearly increasing rank values. In order to make the performance of the fine-tuned model reach or exceed that of full-parameter fine-tuning, so that the generated content better meets the requirements of downstream tasks, we adjust the matrix weights to align them with those of full-parameter fine-tuning. To mitigate the increased computational load and resource consumption caused by the increase in rank values and gradient alignment, we incorporate two pruning methods to handle large models of different scales. Therefore, we propose BladeLoRA, which consists of three parts. First, we design an increasing sequence of rank values. Then, we adjust the weights of specific layers to approximate the results of full-parameter fine-tuning. Finally, we apply different algorithmic pruning techniques to pre-trained large models of different scales. We conduct experiments on the T5 and Llama2 models, and the experimental results fully validate the effectiveness of BladeLoRA.