<p>Speaker diarization, which is the act of segmenting audio to identify different speakers, has significant challenges in overlapping speech situations because traditional methods such as clustering and x-vectors sometimes lose accuracy. This study explores the use of Large Language Models (LLMs), particularly GPT-J, in conjunction with conventional acoustic-based diarization algorithms to enhance speaker distinction in complex conversational scenarios. The system's contextual understanding of the discussion significantly improves speaker segmentation during overlapping speech occurrences. An LLM is used to process the output once Automatic Speech Recognition (ASR) is acquired for transcription. Experimental results show that the combination of aural signals with the semantic capabilities of LLMs lowers the rate of diarization errors. The proposed hybrid technique has a variety of real-time transcription applications for customer support, meetings, and multi-speaker environments. In addition, it shows promising advancements in diarization. Further work will extend to improving ASR performance and studying the design of more advanced LLMs in order to strengthen the approach on various datasets and scenarios. Preliminary results are shown to indicate significant improvement in diarization accuracy and error reduction rates when compared to exclusively acoustic-based methods. This hybrid technique would increase speaker diarization's dependability by quite a significant margin, especially in real-world conditions where the system is to deliver an accurate transcription along with speaker identification. Further research opportunities also lie within optimizing further such as ASR performance improvement and experimentation with more complex LLM structures for improving system robustness over various datasets and scenarios. This work lays the foundation for more accurate and contextually sensitive diarization models, thus opening new opportunities to audio process by multi-speaker real-time streams.</p>