Language Modeling and Large Language Models
摘要
This chapter surveys the conceptual and technical scaffolding of modern large language models (LLMs) and their impact on language services. It first problematizes “computable linguistic units,” showing how segmentation and tokenization (from words to subwords and bytes) operationalize theory while trading off sparsity, semantic integrity, and engineering efficiency. It then contrasts dynamic sequence modeling with static representation learning, tracing evolution from n‑grams to neural and Transformer architectures and explaining the rise of autoregressive (decoder–only) scaling over masked objectives for unified generation and emergent in-context learning. The pretraining–fine–tuning paradigm has shifted toward prompt–driven interaction, supplemented by continued pretraining, supervised fine‑tuning, knowledge distillation, retrieval–augmented generation, and agent frameworks that orchestrate multistep tool use. Core service-oriented capabilities now hinge on efficient adaptation, controllability, and integration with external knowledge. Technical and socio–ethical limitations—hallucination, bias, responsibility attribution, interpretability, and control—are analyzed as structural consequences of probabilistic next‑token prediction and data provenance rather than isolated defects. The chapter frames LLMs as transformative yet fallible linguistic infrastructures, emphasizing the emerging professional skill set: critical evaluation, prompt and data curation, bias mitigation, and system orchestration for trustworthy language services.