Large Language Models (LLMs) have made a significant mark in the field of natural language processing (NLP), yet their performance often falls short when applied to low-resource languages, such as the Saudi dialect of Arabic. This is especially true in the realm of customer service, where challenges arise in sentiment classification and justification generation. This study tackles these difficulties by fine-tuning three language models; two smaller models (SLMs), Qwen2-0.5B and TinyLlama-1.1B-Chat-v1.0, along with the larger model Qwen-7B using Low-Rank Adaptation (LoRA) on a unique dataset comprising around 8,967 customer service dialogues in Saudi Arabic. The objective is to develop a robust model capable of analyzing lengthy textual customer conversations by accurately assigning sentiment labels (positive, negative, neutral) and providing justifications for each sentence. To address issues such as sentence merging and omission, evaluations are performed at both the dialogue and sentence levels. The findings reveal that the fine-tuned models significantly surpassed their base versions, with the Qwen-7B model showing a 26.2% improvement in sentiment classification accuracy and a 35.4% boost in justification generation, underscoring the successful application of fine-tuning strategies for engaging with Saudi Arabic data in customer service contexts.