LORD: a learned distributed database query optimizer with exploration space adaptive decay search strategy
摘要
With the rapid evolution of big data applications, distributed databases have become critical infrastructure for large-scale data processing, and multi-replica query scenarios are increasingly common. However, existing distributed query optimizers often neglect inter-node communication overhead, failing to capture the combined impact of server processing, disk I/O, and network latency. This oversight hinders the generation of globally optimal query execution plans. Meanwhile, while deep reinforcement learning (DRL)-based optimizers have shown promise in centralized environments, they face severe challenges in distributed environments due to an expanded plan search space and latency uncertainty arising from data distribution and replica management. To address these challenges, we propose LORD, a learned distributed database query optimizer with exploration space adaptive decay search strategy. LORD introduces a communication-aware multi-replica scheduling strategy to reduce query latency, as well as an exploration space adaptive decay search strategy that transitions from broad exploration to fine-grained tuning, thus improving both efficiency and plan quality. Experiments on JOB, TPC-DS, and STACK benchmarks show that LORD achieves 10.27