Integrating Autonomous Driving (AD) and Large Language Models (LLMs) represents a significant advancement in automotive intelligence. However, while LLMs excel by leveraging extensive historical knowledge, they lack real-time and reliable knowledge, which is critical to the dynamic nature of the AD domain, such as continuously changing road conditions that significantly impact autonomous vehicle (AV) decisions by necessitating immediate responses to environmental shifts and hazards. A pressing challenge is the large-scale collection of real-time and reliable knowledge within AD networks and its effective use to guide the generative processes of LLMs, which is still an unresolved issue in the AD field. In this paper, we present the AutoVLLM: real-time and reliability-verifiable LLMs for automotive intelligence. The key idea is to propose a knowledge-consensus-based blockchain to overcome the limitations of LLMs in accessing real-time reliable knowledge while ensuring privacy. First, we propose a knowledge auction-based retrieval-augmentation-generation (RAG) technique to improve the generation of LLMs in the AD domain. Second, to verify the timeliness and reliability of shared knowledge, we present a real-time reliable verification knowledge consensus algorithm, which is a crucial element for informed AD decision-making. Third, experiments demonstrate the effectiveness and superiority of our scheme in enhancing the intelligence of autonomous vehicles.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real-Time Reliable Large Language Models with Distributed Knowledge Crowdsourcing for Automotive Mobile Intelligence

  • Jinhao Zhou,
  • Jun Wu

摘要

Integrating Autonomous Driving (AD) and Large Language Models (LLMs) represents a significant advancement in automotive intelligence. However, while LLMs excel by leveraging extensive historical knowledge, they lack real-time and reliable knowledge, which is critical to the dynamic nature of the AD domain, such as continuously changing road conditions that significantly impact autonomous vehicle (AV) decisions by necessitating immediate responses to environmental shifts and hazards. A pressing challenge is the large-scale collection of real-time and reliable knowledge within AD networks and its effective use to guide the generative processes of LLMs, which is still an unresolved issue in the AD field. In this paper, we present the AutoVLLM: real-time and reliability-verifiable LLMs for automotive intelligence. The key idea is to propose a knowledge-consensus-based blockchain to overcome the limitations of LLMs in accessing real-time reliable knowledge while ensuring privacy. First, we propose a knowledge auction-based retrieval-augmentation-generation (RAG) technique to improve the generation of LLMs in the AD domain. Second, to verify the timeliness and reliability of shared knowledge, we present a real-time reliable verification knowledge consensus algorithm, which is a crucial element for informed AD decision-making. Third, experiments demonstrate the effectiveness and superiority of our scheme in enhancing the intelligence of autonomous vehicles.