In the medical field, with advancements in medical equipment technology and the increasing variety of medical devices, medical data has become more refined and precise. Medical professionals rely on data provided by various medical devices to comprehensively assess patients’ health conditions. However, the continuous growth in data volume poses significant challenges to multi-condition join queries of data tables. Although traditional databases and data warehouses can perform effective and accurate data table join queries, they struggle to support join queries on large-scale data volumes efficiently. To address these issues, we propose an optimization method for multi-condition join query based on data lakes. We initially designs a dynamic join query strategy that selects different join query plans based on the query frequency of data tables, thereby achieving an optimal balance between time and space overhead. Furthermore, by leveraging the advantages of adjacent data aggregation, we construct multi-dimensional indexes for medical data, significantly enhancing the efficiency of multi-condition join query.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-condition Join Query Optimization for Disease-Specific Data Exploration Based on Data Lake

  • Jiahao Li,
  • Wenkui Zheng,
  • Kun Chao,
  • Jiaxu Guo,
  • Tianyi Liu,
  • Kaijun Wen,
  • Yong Zhang

摘要

In the medical field, with advancements in medical equipment technology and the increasing variety of medical devices, medical data has become more refined and precise. Medical professionals rely on data provided by various medical devices to comprehensively assess patients’ health conditions. However, the continuous growth in data volume poses significant challenges to multi-condition join queries of data tables. Although traditional databases and data warehouses can perform effective and accurate data table join queries, they struggle to support join queries on large-scale data volumes efficiently. To address these issues, we propose an optimization method for multi-condition join query based on data lakes. We initially designs a dynamic join query strategy that selects different join query plans based on the query frequency of data tables, thereby achieving an optimal balance between time and space overhead. Furthermore, by leveraging the advantages of adjacent data aggregation, we construct multi-dimensional indexes for medical data, significantly enhancing the efficiency of multi-condition join query.