The COVID-19 pandemic, caused by the novel coronavirus, has swept the world due to its unprecedented nature. To increase understanding of the disease and develop countermeasures, it is essential to collect and store data in an appropriate and effective format. However, the large amount of data obtained from various heterogeneous sources is usually dynamic and rigid, making traditional data warehouses less effective for storing sporadic and constantly changing clinical data related to COVID-19 patients. This paper uses the main dataset obtained from COVID-19 patients to demonstrate the inefficiency of traditional data warehouses and proposes a new architectural model, the data Lakehouse, which combines the key benefits of data lakes and data warehouses to create a single platform. The purpose is to propose conversion rules to migrate from traditional data warehouse databases to modern data Lakehouse solutions. To achieve this, we employ model-driven architecture (MDA) and transformation languages, such as MOF 2.0 QVT (Meta Object Facility 2.0 Query-View-Transform) and Acceleo, which define the meta-model used to develop the transformation model.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Model-Driven Transformation for Implementing Data Lakehouse Systems: Covid-19 Use Case

  • Fatima Zahra Belkadi,
  • Fadwa Bouhafer,
  • Anass Elhaddadi,
  • Bahri Abdelkhalek

摘要

The COVID-19 pandemic, caused by the novel coronavirus, has swept the world due to its unprecedented nature. To increase understanding of the disease and develop countermeasures, it is essential to collect and store data in an appropriate and effective format. However, the large amount of data obtained from various heterogeneous sources is usually dynamic and rigid, making traditional data warehouses less effective for storing sporadic and constantly changing clinical data related to COVID-19 patients. This paper uses the main dataset obtained from COVID-19 patients to demonstrate the inefficiency of traditional data warehouses and proposes a new architectural model, the data Lakehouse, which combines the key benefits of data lakes and data warehouses to create a single platform. The purpose is to propose conversion rules to migrate from traditional data warehouse databases to modern data Lakehouse solutions. To achieve this, we employ model-driven architecture (MDA) and transformation languages, such as MOF 2.0 QVT (Meta Object Facility 2.0 Query-View-Transform) and Acceleo, which define the meta-model used to develop the transformation model.