In this paper, we propose a new data model that uses data lineage, which represents relationships between data and their histories, for the verification of data. In recent years, although the verifiability of processes in many industries for their data is required, current data management systems are inadequate, as they do not distinguish between data flow and data dependencies, and also offer little support for machine learning models. Our data model for managing data lineage metadata is based on the property graph model, and supports not only machine learning models which must be treated as both operations to data as well as the data derived from training datasets, but also database repairs and database simulations, which are for maintaining the integrity of data and leveraging past data for future use.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Data Model of a Data Lineage Management System for Database Repair and Simulation

  • Wei Jun Wong,
  • Kyoko Yasuda,
  • Qiong Chang,
  • Jun Miyazaki

摘要

In this paper, we propose a new data model that uses data lineage, which represents relationships between data and their histories, for the verification of data. In recent years, although the verifiability of processes in many industries for their data is required, current data management systems are inadequate, as they do not distinguish between data flow and data dependencies, and also offer little support for machine learning models. Our data model for managing data lineage metadata is based on the property graph model, and supports not only machine learning models which must be treated as both operations to data as well as the data derived from training datasets, but also database repairs and database simulations, which are for maintaining the integrity of data and leveraging past data for future use.