Scholarlink

This chapter provides an in-depth overview of prominent TransformerTransformer-based language models, focusing primarily on BERTBERT (BidirectionalBidirectional EncoderEncoder RepresentationsRepresentation from Transformers)Transformer and GPTGPT (Generative Pre-trained Transformer)Transformer, two foundational architecturesArchitecture that have significantly shaped modern Natural Language ProcessingNatural language processing (NLP) and Large Language ModelsLarge Language Model (LLM) (LLMs). BERTBERT, developed by Google, is no longer state of the art by 2025; however, it remains influential for its use of stacked TransformerTransformer encodersEncoder to produce context-aware token embeddingsEmbedembedding and for its strong performance on diverse NLP tasksTask, such as sentiment analysis and question answering. GPTGPT, introduced by OpenAI, is a generative modelGenerativegenerative model built on TransformerTransformer decodersDecoder and serves as the backbone for many state-of-the-art LLMs. The chapter further explores T5Text-to-Text Transfer Transformer (T5), a unifiedUnified text-to-text model that bridges the design philosophies of BERTBERT and GPTGPT by casting diverse NLP problems into a single framework. Key advancements such as chain-of-thought prompting are also discussed to illustrate emerging techniques in language understanding and generation. Additionally, the chapter surveys notable LLMs and examines research efforts aimed at explaining why LLMs perform so effectively. It then addresses the critical topic ofAlignment alignment—methods to ensure model outputs align with human values and intentions—detailing strategies such as Supervised Fine-TuningSupervisedsupervised fine-tuning (SFT), Reward Modeling, Reinforcement LearningLearningreinforcement learning from Human FeedbackReinforcement learningreinforcement learning from human feedback (RLHF) (RLHF), and Direct Preference OptimizationDirect Preference Optimization (DPO) (DPO). Familiarity with the attentionAttention mechanism and TransformersTransformer is recommended to fully benefit from this chapter. Note that although large language modelsLarge Language Model (LLM) are generative modelsGenerativegenerative model, they are presented in this chapter—following the discussion of the attentionAttention mechanism—for improved readability and flow.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Large Language Models

  • 章节

  • 发表日期：

    2026年

查看全文
  • Benyamin Ghojogh,
  • Ali Ghodsi

摘要

This chapter provides an in-depth overview of prominent TransformerTransformer-based language models, focusing primarily on BERTBERT (BidirectionalBidirectional EncoderEncoder RepresentationsRepresentation from Transformers)Transformer and GPTGPT (Generative Pre-trained Transformer)Transformer, two foundational architecturesArchitecture that have significantly shaped modern Natural Language ProcessingNatural language processing (NLP) and Large Language ModelsLarge Language Model (LLM) (LLMs). BERTBERT, developed by Google, is no longer state of the art by 2025; however, it remains influential for its use of stacked TransformerTransformer encodersEncoder to produce context-aware token embeddingsEmbedembedding and for its strong performance on diverse NLP tasksTask, such as sentiment analysis and question answering. GPTGPT, introduced by OpenAI, is a generative modelGenerativegenerative model built on TransformerTransformer decodersDecoder and serves as the backbone for many state-of-the-art LLMs. The chapter further explores T5Text-to-Text Transfer Transformer (T5), a unifiedUnified text-to-text model that bridges the design philosophies of BERTBERT and GPTGPT by casting diverse NLP problems into a single framework. Key advancements such as chain-of-thought prompting are also discussed to illustrate emerging techniques in language understanding and generation. Additionally, the chapter surveys notable LLMs and examines research efforts aimed at explaining why LLMs perform so effectively. It then addresses the critical topic ofAlignment alignment—methods to ensure model outputs align with human values and intentions—detailing strategies such as Supervised Fine-TuningSupervisedsupervised fine-tuning (SFT), Reward Modeling, Reinforcement LearningLearningreinforcement learning from Human FeedbackReinforcement learningreinforcement learning from human feedback (RLHF) (RLHF), and Direct Preference OptimizationDirect Preference Optimization (DPO) (DPO). Familiarity with the attentionAttention mechanism and TransformersTransformer is recommended to fully benefit from this chapter. Note that although large language modelsLarge Language Model (LLM) are generative modelsGenerativegenerative model, they are presented in this chapter—following the discussion of the attentionAttention mechanism—for improved readability and flow.