Data quality comprises a large set of dimensions, each describing a specific aspect. The assessment of these dimensions requires the collection of simple statistics, the identification of syntactic problems and factual errors, as well as organizational and business aspects to be considered. With the current trend in data-oriented sciences and the increasing reliance on machine learning methods and AI systems, the challenges of poor data quality are ever more apparent. Even recent legislation, such as the European AI Act, mentions data quality requirements for training data; with it, the notion of data quality extends to novel dimensions, such as fairness, diversity, or explainability. We present a modular approach to the assessment of data quality, based on five key facets: data, source, system, task, and human.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Data Quality in the Age of AI

  • Felix Naumann,
  • Lisa Ehrlinger,
  • Hazar Harmouch,
  • Sedir Mohammed,
  • Divesh Srivastava

摘要

Data quality comprises a large set of dimensions, each describing a specific aspect. The assessment of these dimensions requires the collection of simple statistics, the identification of syntactic problems and factual errors, as well as organizational and business aspects to be considered. With the current trend in data-oriented sciences and the increasing reliance on machine learning methods and AI systems, the challenges of poor data quality are ever more apparent. Even recent legislation, such as the European AI Act, mentions data quality requirements for training data; with it, the notion of data quality extends to novel dimensions, such as fairness, diversity, or explainability. We present a modular approach to the assessment of data quality, based on five key facets: data, source, system, task, and human.