Remote photoplethysmography (rPPG) noninvasively measures physiological parameters by analyzing facial blood flow variations, demonstrating broad prospects in medical monitoring and health management applications. However, existing deep learning methods are limited to single-task estimation (HR or \({\text{SpO}}_{{2}}\) alone), failing to capture the intrinsic correlation between these vital signs. Thus, we present HeartOx, a multi-task deep learning framework capable of simultaneously achieving accurate non-contact estimation of both blood oxygen saturation and heart rate. HeartOx includes a \({\text{SpO}}_{{2}}\) branch and a HR branch and is finally optimized by combing heart rate loss and blood oxygen loss. Moreover, to improve HR estimation, we design a plug-and-play Spatio-temporal SE Attention Stack Conv module (STSE). It enhances key features while reducing noise and redundancy, using stacked spatio-temporal convolutions to better capture rPPG signal dynamics and relationships. The results show that our multi-task model achieves comparable or better results on concurrent HR and \({\text{SpO}}_{{2}}\) estimation, compared with the existing task-specific models.