Understanding cyber threats is crucial for effective defense in the field of cybersecurity. If we can automatically map Common Vulnerabilities and Exposures (CVEs) to attack tactics and techniques, it will help practitioners quickly analyze reports and take responsive actions. In this paper, we introduce CyberLLM, leveraging the tailor-made large language model for mapping CVEs to cyber threat tactics and techniques. Specifically, we model the mapping of CVE to tactics and techniques as a multi-label classification problem, given that many CVEs correspond to multiple techniques of ATT&CK. Then, the text description is vectorized through the tokenization process, and we deploy a series of data augmentation techniques to enrich the semantic information. Considering that external knowledge bases are helpful to enhance the contextual information of the queried CVE, CyberLLM designs a retrieval strategy based on the Jaccard distance calculation. Finally, we support flexible model fine-tuning to adapt to the needs. Through extensive experiments, we demonstrate the superiority of CyberLLM compared with 7 representative state-of-the-art methods. We also perform ablation experiments on data augmentation and evaluate the effectiveness of using retrieval information. Furthermore, we provide a series of deep insights in terms of feature attribution and attention weight visualization.