The speaker diarization task aims to identify “who spoke when.” Traditional approaches primarily involve clustering-based methods and end-to-end systems, which typically focus on improving the accuracy of speaker identification while paying less attention to the role of semantic information. In this paper, we propose a novel approach that constructs dedicated prompts for the speaker diarization task, leveraging large language models (LLMs) to directly generate speaker diarization results, thereby completing the task at the semantic level. This method involves generating time-stamped transcriptions through voice activity detection (VAD) and automatic speech recognition (ASR) systems, followed by prompt construction to create specific prompts. These prompts are then processed by the large language model to generate the final speaker diarization results. We used CDER as the evaluation metric, and experimental results on the MagicData-RAMC dataset show that our method improves performance by 30% compared to the baseline system.