Scholarlink

<p>Fine-tuning large language models for specialized domains poses significant computational challenges that limit their accessibility to researchers with standard hardware configurations. This study evaluates parameter-efficient fine-tuning (PEFT) techniques as practical solutions for adapting LLaMA-2 (7B) and LLaMA-3 (8B-Instruct) models to financial sentiment analysis tasks. Through systematic experiments on the Financial PhraseBank dataset, we show that LLaMA-3 with Low-Rank Adaptation (LoRA) reaches 86.89% accuracy on our evaluation split, surpassing prior results reported for comparable settings. Under identical PEFT protocols, LLaMA-3 attains higher accuracy than LLaMA-2; we treat this gap as an observational, multi-factor result (base-model dependence) rather than a causal effect of any single change (e.g., parameter count or architecture).Because we did not run a same-base full fine-tuning (FFT) baseline due to VRAM limits (≈ 97–112 GB estimated vs. our A100-40 GB), we avoid any implication that PEFT surpasses FFT. In our setup, LoRA reduced training memory to ~ 27–30 GB with ~ 32 tokens/s inference throughput, and QLoRA to ~ 17–18 GB with ~ 24 tokens/s (4-bit quantization). These observations support PEFT as a practical mechanism for resource-constrained environments while maintaining competitive performance. The reduction in trainable parameters from 8B to ~ 33.6&#xa0;M without significant accuracy loss underscores the accessibility benefits of PEFT for real-world deployments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Parameter-efficient fine-tuning of LLaMA models for financial sentiment classification

  • 期刊论文

  • 发表日期：

    2025年11月19日

查看全文
  • Seda BAYAT TOKSOZ,
  • Gültekin ISIK

摘要

Fine-tuning large language models for specialized domains poses significant computational challenges that limit their accessibility to researchers with standard hardware configurations. This study evaluates parameter-efficient fine-tuning (PEFT) techniques as practical solutions for adapting LLaMA-2 (7B) and LLaMA-3 (8B-Instruct) models to financial sentiment analysis tasks. Through systematic experiments on the Financial PhraseBank dataset, we show that LLaMA-3 with Low-Rank Adaptation (LoRA) reaches 86.89% accuracy on our evaluation split, surpassing prior results reported for comparable settings. Under identical PEFT protocols, LLaMA-3 attains higher accuracy than LLaMA-2; we treat this gap as an observational, multi-factor result (base-model dependence) rather than a causal effect of any single change (e.g., parameter count or architecture).Because we did not run a same-base full fine-tuning (FFT) baseline due to VRAM limits (≈ 97–112 GB estimated vs. our A100-40 GB), we avoid any implication that PEFT surpasses FFT. In our setup, LoRA reduced training memory to ~ 27–30 GB with ~ 32 tokens/s inference throughput, and QLoRA to ~ 17–18 GB with ~ 24 tokens/s (4-bit quantization). These observations support PEFT as a practical mechanism for resource-constrained environments while maintaining competitive performance. The reduction in trainable parameters from 8B to ~ 33.6 M without significant accuracy loss underscores the accessibility benefits of PEFT for real-world deployments.