Despite the success of generative Large Language Models (LLMs) in tasks like question answering, their direct application to Named Entity Recognition (NER) still struggles to achieve better results than other approaches due to issues such as hallucinations, making them unreliable for direct use. Hallucinations are a consequence of the decoder-only architecture, and although LLMs can mitigate them through a fine-tuning process, this approach can be computationally expensive. To reduce the cost of the fine-tuning process, Parameter-Efficient Fine-tuning (PEFT) techniques such as Low-Rank Adaptation (LoRA) can be used to enable weight adjustments, improving training efficiency. Since LLMs have a massive size, Small Language Models (SLMs) have emerged as compact alternatives, reducing the number of parameters while preserving performance. However, this parameter cutoff diminishes the model’s learning capabilities, particularly in domain-specific tasks, such as legal document NER. In this work, we propose a decoder-only approach to Portuguese legal NER using generative SLMs fine-tuned with LoRA. We assess the impact of LoRA placement within the SLM architecture on performance. Our analysis reveals how LoRA’s arrangement and dimensionality affect training efficacy, yielding a fine-tuned SLM that outperforms prior generative SLMs on legal NER benchmarks.