People often underestimate their carbon emissions, but once they become aware of the environmental impact of their actions, it can lead to significant behavioral changes. Numerous studies have highlighted the importance of improving low-carbon literacy, especially through personalized information, which is more effective in driving behavior change than generalized information. Chatbots can play a key role in delivering tailored messages, providing guidance on food choices, building materials, appliance use, transportation options (e.g., car sharing), and other behaviors that influence an individual’s carbon footprint. This study aims to develop a simple, scalable, and easily maintainable low-carbon activity database using open-source software, operating locally to reduce reliance on paid APIs. Implemented on the Ollama platform, the system can be deployed on consumer-grade computers with acceptable response times. By utilizing a Retrieval-Augmented Generation (RAG) approach, the system converts data into a vector database and uses a large language model for retrieval. Carbon emission data from four sources were used, and the system’s retrieval performance was tested with 100 prompts in English, Traditional Chinese, and Simplified Chinese. The results showed that database size had the largest impact on recall rates, especially when irrelevant or similar data were included. Model selection was also critical, particularly in multilingual contexts where proper language settings played a significant role. The study found that chunk size had a minor influence on results, but the choice of language, while important, was less critical than database size. These findings underscore the need for careful database design and model selection, especially in multilingual RAG systems, and suggest further research to address challenges like scalability and adaptation to diverse data types.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating the Performance of Open-Source LLMs in Local RAG Systems: A Practical Study on Low-Carbon Data Applications

  • Yu-Jie Chang,
  • Tai-Yi Yu,
  • Chung-Hao Chang

摘要

People often underestimate their carbon emissions, but once they become aware of the environmental impact of their actions, it can lead to significant behavioral changes. Numerous studies have highlighted the importance of improving low-carbon literacy, especially through personalized information, which is more effective in driving behavior change than generalized information. Chatbots can play a key role in delivering tailored messages, providing guidance on food choices, building materials, appliance use, transportation options (e.g., car sharing), and other behaviors that influence an individual’s carbon footprint. This study aims to develop a simple, scalable, and easily maintainable low-carbon activity database using open-source software, operating locally to reduce reliance on paid APIs. Implemented on the Ollama platform, the system can be deployed on consumer-grade computers with acceptable response times. By utilizing a Retrieval-Augmented Generation (RAG) approach, the system converts data into a vector database and uses a large language model for retrieval. Carbon emission data from four sources were used, and the system’s retrieval performance was tested with 100 prompts in English, Traditional Chinese, and Simplified Chinese. The results showed that database size had the largest impact on recall rates, especially when irrelevant or similar data were included. Model selection was also critical, particularly in multilingual contexts where proper language settings played a significant role. The study found that chunk size had a minor influence on results, but the choice of language, while important, was less critical than database size. These findings underscore the need for careful database design and model selection, especially in multilingual RAG systems, and suggest further research to address challenges like scalability and adaptation to diverse data types.