Composite materials are increasingly being applied in spacecraft structures. This paper investigates the nonlinear mechanical behavior of a space deployable composite boom (DCB) during the coiling process. A finite element model is established using ABAQUS software, optimizing complex boundary conditions to analyze their dynamic mechanical performance. Through simulations of DCB coiling process, the evolution of strain energy during coiling is revealed, and it is found that the driving torque stabilizes after 0.5 coiling circle. Further parametric analysis indicates the driving torque decreases gradually with the coiling radius increasing. The research results provide theoretical support for the design of DCBs and their applications in space structures.