This chapter mainly introduces the application of large language models in autonomous driving. Section 17.1 introduces the pre-training and fine-tuning of large language models; Sect. 17.2 discusses its emergent capabilities, namely prompting, in-context learning (ICL), instruction following, model alignment, and reasoning of chain-of-thought (CoT); Sect. 17.3 generalizes the large language model to the visual-language domain, namely the vision-large language model; Sect. 17.4 further generalizes it to the multimodal large language model, which is convenient for application in complex scenarios in real environments; Sect. 17.5 introduces the world model that can model and predict, as well as embodied intelligence technology with environmental interaction capabilities; Sect. 17.6 introduces the optimization and scaling of the inference/test-time of the large scale model; Sect. 17.7 introduces the application of the large scale model in autonomous driving with examples; Sect. 17.8 summarizes the contents of this chapter.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Large Scale Models in Autonomous Driving

  • Yu Huang,
  • Zijiang Yang

摘要

This chapter mainly introduces the application of large language models in autonomous driving. Section 17.1 introduces the pre-training and fine-tuning of large language models; Sect. 17.2 discusses its emergent capabilities, namely prompting, in-context learning (ICL), instruction following, model alignment, and reasoning of chain-of-thought (CoT); Sect. 17.3 generalizes the large language model to the visual-language domain, namely the vision-large language model; Sect. 17.4 further generalizes it to the multimodal large language model, which is convenient for application in complex scenarios in real environments; Sect. 17.5 introduces the world model that can model and predict, as well as embodied intelligence technology with environmental interaction capabilities; Sect. 17.6 introduces the optimization and scaling of the inference/test-time of the large scale model; Sect. 17.7 introduces the application of the large scale model in autonomous driving with examples; Sect. 17.8 summarizes the contents of this chapter.