This chapter explores how large language models (LLMs) tackle complex tasks through advanced reasoning strategies, highlighting techniques that extend beyond simple prompts. It first introduces the LLM-based planning framework, outlining how tasks are decomposed into subtasks and refined based on feedback from either external sources like tools and humans or internal assessments provided by the model itself. Building on this foundation, it discusses the construction of LLM-based agent systems, emphasizing three core components: a memory module (for storing and retrieving key information), a planning module (for generating and refining action sequences), and an execution module (for carrying out actions in real or virtual environments). The chapter then illustrates how these agents operate individually and in multi-agent settings, where communication and coordination mechanisms enable them to share information, allocate tasks, and collectively achieve larger goals. Several application examples-including WebGPT for information retrieval, MetaGPT for software development, and Westworld-style simulations-demonstrate the versatility of LLM-based agent systems. Despite promising results, the chapter also underscores major challenges, such as resource consumption, tool integration, communication bottlenecks, and the complexities of operating in real-world scenarios. Concluding with the new paradigm of long chain-of-thought reasoning, this chapter shows how extended internal reflections, iterative verification, and more involved search strategies allow LLMs to solve increasingly difficult problems, thereby laying the groundwork for more sophisticated and autonomous AI systems.