Background: Cross-project defect prediction (CPDP) makes use of historical defect datasets gathered from source projects to train a model, which is then applied to a target project. Large language models (LLMs) have achieved promising performance on many fields including software engineering, e.g., code generation. However, existing cross-project defect prediction (CPDP) methods mainly center on traditional machine learning or deep-learning approaches and fail to explore the potential of large language models (LLMs). It is still an open question that whether LLMs are helpful for CPDP. Objective: To tackle this issue, we put forward a novel approach named LLM4SDP. This approach makes use of large language models to achieve cross-project prediction capabilities. As far as we know, this is the first time that large language models have been applied to cross-project defect prediction (CPDP) in the field of software defect prediction. Method: We investigate large language models including Qwen2-7b, Llama3-8b-instruction, and CodeGemma-7B-Chat. The well-known fine-tuning method LoRA is employed to adjust LLMs to help them better adapt to downstream tasks. We further explored the effect of over-sampling methods on LLM performance owing to the natural class imbalance of software defect datasets. Results: Experiments were conducted on five defect datasets (EQ, JDT, LC, ML, PDE), performance was evaluated using metrics like the Matthews correlation coefficient (MCC). Results show: (1) the proposed method yields varying performance across datasets. On the JDT dataset, the Qwen2-7b model achieves an MCC of 0.40 with SMOTE, a notable improvement. However, SMOTE decreases performance on EQ and ML. (2) SMOTE improves recall, especially in imbalanced datasets, but may reduce precision. (3) compared to baseline methods, the proposed approach demonstrates significant improvements in some datasets. Conclusion: It can be concluded that: (1) applying data balancing techniques like SMOTE is beneficial in certain datasets, especially for improving recall in imbalanced datasets (e.g., JDT, LC); (2) the proposed method shows promising results in some scenarios compared with existing CPDP methods, particularly with the CodeGemma-7B-Chat.