Energy Efficiency in Large Language Models: An Empirical Study
摘要
The energy efficiency of large language models (LLMs) during inference is a critical area of research, driven by the increasing integration of these models across various practical applications. This study examines the energy consumption dynamics of LLMs, focusing on the relationship between batch sizes, model configurations, and energy usage. As demand for computational resources such as GPUs grows, it leads to heightened energy use, raising both environmental and economic concerns. To address these challenges, we leverage the vLLM framework, known for its efficient batch handling, to assess how different batch sizes affect text generation speed and energy consumption. Our findings demonstrate that, while maximizing batch sizes enhances token throughput and reduces energy consumption per token, it can also impact response times. By exploring the use of advanced techniques such as continuous batching, our study provides insights into optimizing LLM deployments for better energy efficiency and sustainability.