Scholarlink

<p>We introduce BIG-SAVE and BIG-DR, novel sufficient dimension reduction methods specifically designed to handle massive datasets, that challenge conventional computation. Our approaches extend traditional sufficient dimension reduction by adopting a divide-and-conquer strategy: the full dataset is partitioned into manageable chunks, sufficient dimension reduction is performed independently on each subset, and the resulting subspaces are integrated using a proximity-based aggregation method. Unlike existing sufficient dimension reduction techniques, BIG-SAVE and BIG-DR are optimized for parallel computing and memory mapping, allowing them to process data that exceed the limits of available random access memory. Moreover, our approach ensures robust estimation stability across chunks and maintains exhaustiveness, ensuring full recovery of the central subspace under mild conditions. We demonstrate the practical performance of our proposed methods through extensive simulation studies and real-world data applications. Our results show that BIG-SAVE and BIG-DR achieve competitive estimation accuracy, significant computational efficiency, and scalability making them suitable for modern large-scale data analysis tasks in scientific and industrial domains.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

On exhaustive sufficient dimension reduction methods for massive datasets

  • 期刊论文

  • 发表日期：

    2026年7月2日

查看全文
  • Chaehyun Ryu,
  • Kyungjin Lee,
  • Kyongwon Kim

摘要

We introduce BIG-SAVE and BIG-DR, novel sufficient dimension reduction methods specifically designed to handle massive datasets, that challenge conventional computation. Our approaches extend traditional sufficient dimension reduction by adopting a divide-and-conquer strategy: the full dataset is partitioned into manageable chunks, sufficient dimension reduction is performed independently on each subset, and the resulting subspaces are integrated using a proximity-based aggregation method. Unlike existing sufficient dimension reduction techniques, BIG-SAVE and BIG-DR are optimized for parallel computing and memory mapping, allowing them to process data that exceed the limits of available random access memory. Moreover, our approach ensures robust estimation stability across chunks and maintains exhaustiveness, ensuring full recovery of the central subspace under mild conditions. We demonstrate the practical performance of our proposed methods through extensive simulation studies and real-world data applications. Our results show that BIG-SAVE and BIG-DR achieve competitive estimation accuracy, significant computational efficiency, and scalability making them suitable for modern large-scale data analysis tasks in scientific and industrial domains.