Scholarlink

<p>The rapid advancement of large language models (LLMs) has driven remarkable progress in natural language processing, yet it has also imposed significant computational, memory, and energy costs. Pruning has emerged as a key model compression technique to mitigate these challenges by selectively removing redundant parameters or components while preserving performance. In this systematic literature review (SLR), we provide a comprehensive overview of pruning techniques applied to LLMs, based on 60 peer-reviewed studies and preprints published between 2022 and 2025, sourced from major digital libraries including IEEE Xplore, ACM Digital Library, ScienceDirect, SpringerLink, arXiv, Wiley Online Library, and Taylor &amp; Francis Online. Our analysis categorizes pruning methods into structured (55%), unstructured (23%), and semi-structured (22%) paradigms, with structured pruning dominating due to its superior hardware compatibility and direct impact on model size. Evaluations were predominantly conducted in Python-based environments using frameworks such as PyTorch and Hugging Face. The most frequently employed datasets include WikiText (18%), C4 (11%), and commonsense/QA benchmarks, while accuracy (39%) and perplexity (28%) remain the primary metrics for assessing pruned model effectiveness. LLaMA (53%) and OPT (12%) are the most commonly targeted models. These findings highlight current trends in LLM pruning and identify key research gaps, paving the way for future advancements in efficient and sustainable AI systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Pruning large language models: a systematic literature review

  • 期刊论文

  • 发表日期：

    2026年7月30日

查看全文
  • Fatemeh Bazikar,
  • Atefeh Hemmati,
  • Akram Reza,
  • Davood Farajzade

摘要

The rapid advancement of large language models (LLMs) has driven remarkable progress in natural language processing, yet it has also imposed significant computational, memory, and energy costs. Pruning has emerged as a key model compression technique to mitigate these challenges by selectively removing redundant parameters or components while preserving performance. In this systematic literature review (SLR), we provide a comprehensive overview of pruning techniques applied to LLMs, based on 60 peer-reviewed studies and preprints published between 2022 and 2025, sourced from major digital libraries including IEEE Xplore, ACM Digital Library, ScienceDirect, SpringerLink, arXiv, Wiley Online Library, and Taylor & Francis Online. Our analysis categorizes pruning methods into structured (55%), unstructured (23%), and semi-structured (22%) paradigms, with structured pruning dominating due to its superior hardware compatibility and direct impact on model size. Evaluations were predominantly conducted in Python-based environments using frameworks such as PyTorch and Hugging Face. The most frequently employed datasets include WikiText (18%), C4 (11%), and commonsense/QA benchmarks, while accuracy (39%) and perplexity (28%) remain the primary metrics for assessing pruned model effectiveness. LLaMA (53%) and OPT (12%) are the most commonly targeted models. These findings highlight current trends in LLM pruning and identify key research gaps, paving the way for future advancements in efficient and sustainable AI systems.