In this chapter, we discuss the framework of the perception-action loop in the embodied multi-agent system for both the homogeneous multi-agent system and the heterogeneous multi-agent system, respectively. The basic structure of the perception-action loop includes visual perception, state feature extraction, communication, and action decision modules. The loop also has different detailed designs according to the characteristics of different tasks. We first analyze the centralized perception-action loop structure in the autonomous task assignment. We propose a dynamic, collaborative task planning framework that utilizes external knowledge sources with dynamically perceived visual information to resolve the high-level instructions and dynamically allocates the decomposed tasks to multiple agents. Next, we discuss the decentralized perception-action loop structure in collaborative object search. We develop a hierarchical decision framework based on semantic mapping, scene prior knowledge, and communication mechanism to solve this task. Then, we analyze the perception-action loop structure in the heterogeneous embodied multi-agent system in the room tidying-up task, in which heterogeneous agents collaborate with others to detect misplaced objects and place them in reasonable locations. The heterogeneous perception-action loop needs to consider the different capabilities of agents when making action decisions. We introduce a hierarchical decision model based on misplaced object detection, reasonable receptacle prediction, and handshake-based group communication mechanism for heterogeneous multi-agent room tidying-up. We also demonstrate simulation experiments of these perception-action frameworks.