<p>Coordinating time-sensitive deliveries in environments like hospitals poses a complex challenge, particularly when managing multiple online pickup and delivery requests within strict time windows using a team of heterogeneous robots. Traditional approaches fail to address dynamic rescheduling or diverse service requirements, typically restricting robots to single-task types. This paper tackles the Multi-Pickup and Delivery Problem with Time Windows (MPDPTW), where autonomous mobile robots are capable of handling varied service requests. The objective is to minimize late delivery penalties while maximizing task completion rates. To achieve this, we propose a novel framework leveraging a heterogeneous robot team and an efficient dynamic scheduling algorithm. Users submit requests with specific time constraints, and our decentralized algorithm–Heterogeneous Mobile Robots Online Diverse Task Allocation (HMR-ODTA)–optimizes task assignments to ensure timely service while addressing delays or task rejections. Extensive simulations validate the algorithm’s effectiveness. For smaller task sets (40–160 tasks), the proposed algorithm reduced penalties by approximately 72% and 99% compared to the Effective and Efficient Performance Impact (EEPI) and the Genetic Algorithm-based Multi-Robot Task Allocation (GA-MR), respectively. For larger task sets (160–280 tasks), the penalties decreased by around 33% and 70% over EEPI and GA-MR, respectively. These results highlight the algorithm’s effectiveness in improving task scheduling and coordination in multi-robot systems, offering a robust solution for enhancing delivery performance in structured, time-critical environments.</p>