Modular robots can adapt to different tasks by assembling into various configurations. Parallely executing the self-assembly process can significantly enhance its efficiency. However, the implementation of parallel assembly presents challenges, as it requires not only accurate mapping of modules between configurations, but also simultaneous control of the robot’s motion. A hierarchical framework is proposed for the parallel self-assembly of modular robots, capable of accommodating varying numbers of robots and assembling them into different target configurations. The framework comprises a mapping algorithm to determine the reconfiguration scheme between the initial and target configurations, as well as a transferable motion model based on SAC (Soft Actor-Critic). The mapping algorithm computes the optimal module position assignments, while the motion model provides an efficient motion strategy for the modular robots. We conducted simulations and experiments on the Webots platform to verify the proposed framework, and the experimental results show that it produces a high-quality parallel self-assembly scheme.