In recent years, space-air-ground integrated network(SAGIN) has been recognized as a promising area in 6G research. In the air-ground component of SAGIN, airborne device such as unmanned aerial vehicles (UAVs) can provide task offloading and computing services to ground devices. Edge servers are installed on UAVs to provide data offloading services for ground devices. The high mobility of UAVs, compared to base stations and edge computing devices on the ground, makes it better able to provide timely and effective services to the devices. Considering the needs of delay-sensitive devices, this paper jointly allocates computational resources and designs UAV trajectories to achieve the goal of minimizing delay. In this paper, we use non-orthogonal multiple access (NOMA) technique in the uplink channel, which allows a UAV to serve multiple ground devices simultaneously. Both the UAV and the ground devices are moving, and the ground devices need to re-establish their connection to the UAV every once in a while. Based on the traditional Deep Reinforcement Learning (DRL) algorithm, this study proposes the Multi-Agent DRL (MADRL) algorithm to jointly determine the optimal 3D trajectory and computational resource allocation of UAVs.The MADRL algorithm achieves complete ground cooperation among multiple UAVs as agents in optimizing the latency by co-training the neural network, simplifying the network structure, and improving the training efficiency. Numerical results show that the proposed MADRL algorithm can converge under the system quality of service (QoS) constraints, and the convergence speed is faster than that of the traditional deep Q network (DQN) algorithm. The average total delay of the system can also be effectively reduced and converged in a multi-UAV scenario.