This chapter begins by defining what is meant by narrative, indicating the scope of what is included in this book. It then introduces the fundamental science of Information Theory, which lays the foundation for understanding the technical developments with LLMs. Among the topics introduced are Zipf’s Law (a power law which is reflected in lots of phenomena including narrative), and entropy, which measures how unpredictable a text is. After explaining how to calculate entropy and the probabilities of sentences, and the ideas behind language models, the chapter discusses the predictability of different written languages like English and Chinese. This overview then naturally leads to larger questions about meaning, where the chapter introduces the linguistic theory of meaning that informs the LLMs and the NLP in use today, namely Distributional Semantics. The chapter then turns to the vexing question of whether a machine actually understands the meaning of the data it’s been fed, including prompts from a human. This is followed by a comparison of deep learning models and human linguistic abilities in relation to both natural language and narrative, after which the limitations of current AI approaches are outlined. After describing the book’s methodological approach, which involves a narrative annotation scheme called NarrativeML, the chapter provides a brief overview of the rest of the book.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Introduction

  • Inderjeet Mani

摘要

This chapter begins by defining what is meant by narrative, indicating the scope of what is included in this book. It then introduces the fundamental science of Information Theory, which lays the foundation for understanding the technical developments with LLMs. Among the topics introduced are Zipf’s Law (a power law which is reflected in lots of phenomena including narrative), and entropy, which measures how unpredictable a text is. After explaining how to calculate entropy and the probabilities of sentences, and the ideas behind language models, the chapter discusses the predictability of different written languages like English and Chinese. This overview then naturally leads to larger questions about meaning, where the chapter introduces the linguistic theory of meaning that informs the LLMs and the NLP in use today, namely Distributional Semantics. The chapter then turns to the vexing question of whether a machine actually understands the meaning of the data it’s been fed, including prompts from a human. This is followed by a comparison of deep learning models and human linguistic abilities in relation to both natural language and narrative, after which the limitations of current AI approaches are outlined. After describing the book’s methodological approach, which involves a narrative annotation scheme called NarrativeML, the chapter provides a brief overview of the rest of the book.