While Large Language Models (LLMs) have demonstrated exceptional performance across various domains, their potential applications in professional sports psychology consulting remain insufficiently evaluated. Existing assessment frameworks predominantly focus on general counseling capabilities or medical expertise, failing to effectively capture the specialized demands and challenges unique to sports psychology. To address this gap, we present SPPBench, a comprehensive evaluation benchmark for assessing LLMs’ performance in sports psychological counseling contexts. The benchmark carefully selects psychological challenges commonly encountered in ten major ball sports, categorizing them into three difficulty levels - basic, intermediate, and advanced - to reflect the diverse psychological demands in competitive sports. SPPBench introduces an innovative assessment framework encompassing five key dimensions: explainability, actionability, emotional support, personalization, and professionalism. This multidimensional framework aims to examine models’ comprehension of fundamental sports psychology principles and their capability to provide practical, sport-specific interventions. These dimensions are systematically integrated into evaluation criteria to ensure comprehensive coverage of typical psychological intervention scenarios in competitive ball sports. The assessment results of current mainstream LLMs using SPPBench reveal significant variations in their ability to provide appropriate psychological support in sports contexts, particularly demonstrating notable limitations in handling complex emotional states and developing targeted athletic intervention strategies. The proposed benchmark not only provides crucial insights into evaluating the current state of LLM applications in sports psychology but also establishes a scientific and efficient foundation for developing automated sports psychological support systems in the future. The dataset is publicly available at https://github.com/justonly-0129/SPPBench .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SPPBench: A Comprehensive Sports Psychology Performance Benchmark for LLMs

  • Xiangyu Guo,
  • Yunzhi Xu

摘要

While Large Language Models (LLMs) have demonstrated exceptional performance across various domains, their potential applications in professional sports psychology consulting remain insufficiently evaluated. Existing assessment frameworks predominantly focus on general counseling capabilities or medical expertise, failing to effectively capture the specialized demands and challenges unique to sports psychology. To address this gap, we present SPPBench, a comprehensive evaluation benchmark for assessing LLMs’ performance in sports psychological counseling contexts. The benchmark carefully selects psychological challenges commonly encountered in ten major ball sports, categorizing them into three difficulty levels - basic, intermediate, and advanced - to reflect the diverse psychological demands in competitive sports. SPPBench introduces an innovative assessment framework encompassing five key dimensions: explainability, actionability, emotional support, personalization, and professionalism. This multidimensional framework aims to examine models’ comprehension of fundamental sports psychology principles and their capability to provide practical, sport-specific interventions. These dimensions are systematically integrated into evaluation criteria to ensure comprehensive coverage of typical psychological intervention scenarios in competitive ball sports. The assessment results of current mainstream LLMs using SPPBench reveal significant variations in their ability to provide appropriate psychological support in sports contexts, particularly demonstrating notable limitations in handling complex emotional states and developing targeted athletic intervention strategies. The proposed benchmark not only provides crucial insights into evaluating the current state of LLM applications in sports psychology but also establishes a scientific and efficient foundation for developing automated sports psychological support systems in the future. The dataset is publicly available at https://github.com/justonly-0129/SPPBench .