Large Language Model (LLM) has shown excellent performance in natural language processing tasks. In this process, deploying LLM on resource-constrained terminal devices still faces huge challenges. Faced with the growing requirements for user data privacy protection, the existing framework has problems such as insufficient scalability and high risk of data privacy leakage. Therefore, this paper proposes an edge-assisted large language model distributed collaborative training framework based on embedded data processing unit (DPU), combining federated learning and split learning architecture to further improve the generalization ability of large language models. In view of the large number of parameter exchange processes in the framework, a DPU-based in-network aggregation method is designed. DPU is deployed in the switch to aggregate model parameters while transmitting data. At the same time, a split learning algorithm based on federated learning is designed to decompose a complete LLM into a server-side model (SSM) and a device-side model (DSM) (deployed on distributed mobile terminals with limited resources). The computational and memory overhead of the forward and backward propagation processes of LLM are modeled, and finally a computationally efficient Transformer block splitting strategy is obtained. In order to further reduce the computational pressure of each terminal, a LLM fine-tuning method based on federated knowledge distillation is designed.