ETL Orchestration with Apache Airflow
摘要
The OneShop data engineering team has been on quite a journey. After successfully building their data lakehouse infrastructure in previous chapters, they now face a new challenge: automation. The team is tired of manually running scripts and processes every time they need to update their analytics. As Maya, the lead data engineer, puts it: "We're spending too much time babysitting our data pipelines when we could be building new features."