<p>In large-scale data analytics systems such as cloud data warehouses, workloads are inherently complex, involving a vast number of rows and columns, and evolve over time, leading to excessive data scanning. Optimized data layout can skip a large number of irrelevant partitions, thereby improving scan performance. Nowadays, numerous static optimized approaches have been proposed for offline layout optimization based on historical workloads, but they struggle to maintain performance under unpredictable workload shifts. Online layout reorganization can address such issues by dynamically adapting data layouts to workload evolution. However, effectively and efficiently adopting dynamic adaptation strategies for online layout reorganization remains challenging in the face of evolving workloads, which arises from the limitations of existing dynamic adaptation approaches in precisely identifying switching moments and accurately selecting target layouts. These limitations lead to delayed or inappropriate layout changes, resulting in limited query performance gains and excessive reorganization cost. In this paper, we propose the framework STAR, a Long <InlineEquation ID="IEq1"><EquationSource Format="TEX">$\underline{{\textbf {S}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">S</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>hort-<InlineEquation ID="IEq2"><EquationSource Format="TEX">$\underline{{\textbf {T}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">T</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>erm Workload-<InlineEquation ID="IEq3"><EquationSource Format="TEX">$\underline{{\textbf {A}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">A</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>ware Optimizer for Online Layout <InlineEquation ID="IEq4"><EquationSource Format="TEX">$\underline{{\textbf {R}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">R</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>eorganization. STAR contains two methods: the <InlineEquation ID="IEq5"><EquationSource Format="TEX">$\underline{{\textbf {D}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">D</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>ual-<InlineEquation ID="IEq6"><EquationSource Format="TEX">$\underline{{\textbf {A}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">A</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>daptive <InlineEquation ID="IEq7"><EquationSource Format="TEX">$\underline{{\textbf {R}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">R</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>eward-Threshold <InlineEquation ID="IEq8"><EquationSource Format="TEX">$\underline{{\textbf {A}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">A</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>wareness method (DARA) and the <InlineEquation ID="IEq9"><EquationSource Format="TEX">$\underline{{\textbf {Du}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">Du</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>al-Source <InlineEquation ID="IEq10"><EquationSource Format="TEX">$\underline{{\textbf {E}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">E</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>nsemble Hybrid-Probabili<InlineEquation ID="IEq11"><EquationSource Format="TEX">$\underline{{\textbf {t}}}$</EquationSource><EquationSource Format="MATHML"><math><munder><mi mathvariant="bold">t</mi><mo>̲</mo></munder></math></EquationSource></InlineEquation>y Switching method (DUET). DARA overcomes the limitation in precise switching moment awareness and DUET addresses the limitation in accurate switching selection. In addition, STAR introduces an optimized initial layout to reduce reorganization cost. Experiments on real-world datasets show that STAR achieves up to 27.7% improvement in combined query and reorganization time over SOTA, and up to a 63.4% decrease in reorganization cost.</p>

STAR: A Long short-term workload-aware optimizer for online layout reorganization

期刊论文
出版模式：

开放获取
发表日期：

2026年7月9日

查看全文

Xianbo Liu,
Yiwen Gao,
Ming Sheng,
Yi Luo,
Zeming Li,
Yong Zhang,
Yuhang Hu

摘要

In large-scale data analytics systems such as cloud data warehouses, workloads are inherently complex, involving a vast number of rows and columns, and evolve over time, leading to excessive data scanning. Optimized data layout can skip a large number of irrelevant partitions, thereby improving scan performance. Nowadays, numerous static optimized approaches have been proposed for offline layout optimization based on historical workloads, but they struggle to maintain performance under unpredictable workload shifts. Online layout reorganization can address such issues by dynamically adapting data layouts to workload evolution. However, effectively and efficiently adopting dynamic adaptation strategies for online layout reorganization remains challenging in the face of evolving workloads, which arises from the limitations of existing dynamic adaptation approaches in precisely identifying switching moments and accurately selecting target layouts. These limitations lead to delayed or inappropriate layout changes, resulting in limited query performance gains and excessive reorganization cost. In this paper, we propose the framework STAR, a Long $\underline{{\textbf {S}}}$ $\underset{̲}{S}$ hort-$\underline{{\textbf {T}}}$ $\underset{̲}{T}$ erm Workload-$\underline{{\textbf {A}}}$ $\underset{̲}{A}$ ware Optimizer for Online Layout $\underline{{\textbf {R}}}$ $\underset{̲}{R}$ eorganization. STAR contains two methods: the $\underline{{\textbf {D}}}$ $\underset{̲}{D}$ ual-$\underline{{\textbf {A}}}$ $\underset{̲}{A}$ daptive $\underline{{\textbf {R}}}$ $\underset{̲}{R}$ eward-Threshold $\underline{{\textbf {A}}}$ $\underset{̲}{A}$ wareness method (DARA) and the $\underline{{\textbf {Du}}}$ $\underset{̲}{Du}$ al-Source $\underline{{\textbf {E}}}$ $\underset{̲}{E}$ nsemble Hybrid-Probabili$\underline{{\textbf {t}}}$ $\underset{̲}{t}$ y Switching method (DUET). DARA overcomes the limitation in precise switching moment awareness and DUET addresses the limitation in accurate switching selection. In addition, STAR introduces an optimized initial layout to reduce reorganization cost. Experiments on real-world datasets show that STAR achieves up to 27.7% improvement in combined query and reorganization time over SOTA, and up to a 63.4% decrease in reorganization cost.