LLMEPBench: A Benchmark for LLMs on Various Electric Power System Application Tasks
摘要
In recent years, the development of large language models (LLMs) has accelerated, leading to a myriad of applications across diverse fields. However, the electric power industry has yet to fully tap into the potential of these models. To address this gap, we introduce LLMEPBench, a dedicated benchmark for LLMs tailored to various tasks within electric power systems. Our benchmark evaluates several well-known LLM options, including Qwen, Wenxin Yiyan, and GLM-4-32B, across multiple tasks such as rating prediction, sequential recommendations, direct recommendations, explanation generation, and review summarization. Additionally, we have compiled a comprehensive data set to accurately measure model performance. Through our qualitative evaluations, we aim to assess the content quality produced by different models. Initial findings indicate that LLMs possess the ability to comprehend the information provided and generate responses that are not only clear but also logically sound. We hope that this benchmark inspires researchers to explore the untapped potential of LLMs in enhancing various applications within the electric power sector.