<p>Recent advances in large language models (LLMs) have produced impressive fluency, yet their application to specialized scientific domains like wood science remains limited. This study introduces WoodLLaMA, a domain-specific LLM fine-tuned on metadata from 16,929 wood science research articles, and examines the effects of fine-tuning and retrieval-augmented generation (RAG) on model performance. Evaluation utilized two datasets not included in the training data: a Journal question–answer (QA) set representing domain-specific expertise and a Wood Handbook QA set reflecting fundamental wood science knowledge. Using intrinsic metrics (perplexity) and QA-based metrics (cosine similarity, keyword matching, and BERTScores), along with qualitative case studies, fine-tuning was found to enhance linguistic fluency while RAG improved semantic alignment. Combining fine-tuning and RAG yielded the most robust and consistent performance. These results demonstrate the complementary value of fine-tuning and RAG for building domain-specific LLMs. The study offers a methodological framework for LLM evaluation and identifies future directions—such as leveraging full-text data, enabling multilingual support, integrating multimodal resources, and incorporating human-in-the-loop learning methods—for enhancing the performance and broadening the applicability of WoodLLaMA across a diverse range of domains.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating fine-tuning and retrieval-augmented generation for domain-specific language modeling in wood science

  • Sung-Wook Hwang,
  • Won-Hee Lee,
  • Bonwook Koo

摘要

Recent advances in large language models (LLMs) have produced impressive fluency, yet their application to specialized scientific domains like wood science remains limited. This study introduces WoodLLaMA, a domain-specific LLM fine-tuned on metadata from 16,929 wood science research articles, and examines the effects of fine-tuning and retrieval-augmented generation (RAG) on model performance. Evaluation utilized two datasets not included in the training data: a Journal question–answer (QA) set representing domain-specific expertise and a Wood Handbook QA set reflecting fundamental wood science knowledge. Using intrinsic metrics (perplexity) and QA-based metrics (cosine similarity, keyword matching, and BERTScores), along with qualitative case studies, fine-tuning was found to enhance linguistic fluency while RAG improved semantic alignment. Combining fine-tuning and RAG yielded the most robust and consistent performance. These results demonstrate the complementary value of fine-tuning and RAG for building domain-specific LLMs. The study offers a methodological framework for LLM evaluation and identifies future directions—such as leveraging full-text data, enabling multilingual support, integrating multimodal resources, and incorporating human-in-the-loop learning methods—for enhancing the performance and broadening the applicability of WoodLLaMA across a diverse range of domains.