Scholarlink

<p>Underground public space (UPS) is vital for three-dimensional urban development, and its post-occupancy evaluation (POE) forms a basis for human-centric planning, design and management. Although large language models (LLMs) provide powerful tools for POEs based on social media data (SMD), LLM application sare often constrained by lacking domain-specific knowledge. This study developed a framework to optimize LLM performance for SMD-based POEs. Model selection, parameter setting, prompt tuning and fine-tuning were examined for two specialized annotation tasks using 34 UPS cases and 14 LLMs. The methods proved satisfactory effects, achieving peak macro accuracy of approximately 0.95 and F1-scores over 0.83 for the two tasks. Reasoning models outperformed general-purpose models by up to 25.74% (F1-scores) in complex tasks, though with lower consistency. Prompt tuning was highly effective but task-dependent with a peak increment of 21.99% in macro F1-scores. Parameter settings and system role modifying had minimal effects. Excessively long prompts may reduce performance. Fine-tuning significantly improved performance and reduced prompt dependency. POEs identified spatial form, wayfinding and operational management as high-priority factors for future design and renovation of UPS. The findings offered a practical and cost-effective optimization pathway for applying LLMs to other similar textual analysis in underground space studies.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

How to optimize large language model performance for human-centric post-occupancy evaluation of underground public space: From model selection to model tuning

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月21日

查看全文
  • Chen-Xiao Ma,
  • Fang-Le Peng,
  • Zi-Jian Li,
  • Yong-Kang Qiao

摘要

Underground public space (UPS) is vital for three-dimensional urban development, and its post-occupancy evaluation (POE) forms a basis for human-centric planning, design and management. Although large language models (LLMs) provide powerful tools for POEs based on social media data (SMD), LLM application sare often constrained by lacking domain-specific knowledge. This study developed a framework to optimize LLM performance for SMD-based POEs. Model selection, parameter setting, prompt tuning and fine-tuning were examined for two specialized annotation tasks using 34 UPS cases and 14 LLMs. The methods proved satisfactory effects, achieving peak macro accuracy of approximately 0.95 and F1-scores over 0.83 for the two tasks. Reasoning models outperformed general-purpose models by up to 25.74% (F1-scores) in complex tasks, though with lower consistency. Prompt tuning was highly effective but task-dependent with a peak increment of 21.99% in macro F1-scores. Parameter settings and system role modifying had minimal effects. Excessively long prompts may reduce performance. Fine-tuning significantly improved performance and reduced prompt dependency. POEs identified spatial form, wayfinding and operational management as high-priority factors for future design and renovation of UPS. The findings offered a practical and cost-effective optimization pathway for applying LLMs to other similar textual analysis in underground space studies.