Congenital Heart Disease (CHD) is the most common birth defect and has a significant impact on healthcare (HC). About 30% of these children require at least one cardiac procedure during their first years of life. An accurate estimation of the intensive care unit length of stay (ICULOS) after surgery could improve HC resources allocation. However, tools to accurately estimate ICULOS for these patients are insufficient. Thus, this study aims to predict ICULOS in pediatric CHD patients undergoing cardiac surgery by employing the Medic-BERT transformer model. Data from pediatric population with CHD undergoing cardiac surgery between September 2020 and September 2023 was collected. The Medic-BERT model was adapted to integrate static (demographic and perioperative data) and dynamic data (vital signs and lab tests) acquired during the first 8 h in the ICU at irregular time points. Then, the model was fine-tuned to solve the desired task (75% of the data was used for training, 12.5% for validation and 12.5% for testing). Additionally, the latent space reached after pooling the embeddings was explored. Preliminary results show that the Medic-BERT model achieved a mean absolute error of 3.6 days when predicting the ICULOS on the test subset. Additionally, the exploration of the model’s latent space revealed insightful data patterns, enhancing the model’s explainability. Our study demonstrates Medic-BERT’s ability to integrate static and dynamic data to effectively predict ICULOS in pediatric CHD patients following cardiac surgery.