Foundation models, such as GPT-3 and BERT, are large-scale AI models pre-trained on vast datasets, enabling them to perform diverse tasks with high accuracy. This chapter traces the evolution from Pre-trained Language Models (PLMs) to Large Language Models (LLMs) and Multimodal Foundation Models (MFMs), detailing their architectures, pre-training strategies, data construction, modeling efficiency, and key intuitions like scaling laws. Moreover, in the section on “Multimodal Foundation Models”, we not only review several cutting-edge multimodal large language models (MLLMs) but also discuss the future-oriented unified multimodal understanding and generation models, embodied multimodal foundation models, and world models. These foundation models form the core of Interactive NLP (iNLP), making broad and versatile interactions with various entities possible due to their universal and powerful capabilities.