This study presents a novel paradigm of clinical reasoning that combines Chain of thought (CoT) reasoning with a refined Large Language Models (LLMs) like Large Language Model Meta AI (LLaMA) and Gemma. The need for precise and comprehensible AI-powered clinical decision support systems is rising in the current digital healthcare environment. Driven by the goal of improving diagnostic accuracy while lowering processing demands, we suggest a new framework that combines an improved LLaMA model alongside its counterpart Gemma model with chain-of-thought (CoT) reasoning. Our approach uses parameter-efficient fine-tuning using Low-Rank Adaptation (LoRA) in conjunction with 4-bit quantization to tackle the problem of efficiency and performance balance. By adding intermediate reasoning processes to a structured medical dataset, we allow the model to systematically advance from evaluating differential diagnoses to drawing a final judgment supported by evidence. Utilizing Transformer Reinforcement Learning (TRL’s) Supervised Fine Tuning Trainer (SFTTrainer) and the unsloth framework, our method accomplishes efficient fine-tuning even on hardware with restricted resources. According to experimental findings, adding CoT improves the model outputs transparency and dependability, bridging the gap between general-purpose LLMs and domain-specific models while providing a scalable clinical decision support solution.