LSTM-Based Query Performance Optimization in LSM-Trees
摘要
Recently, advancements in storage architectures and query optimization techniques have significantly improved the performance of database, driving the evolution of database systems. Moreover, the demand for database systems to quickly index and retrieve large volumes of data has increased substantially. The log-structured merge tree (LSM-tree), recognized for its excellent write performance, is particularly well-suited for NoSQL database due to its capability to handle large datasets and facilitate high-frequency writes and updates to indexes and data. However, the multi-level storage structure of LSM-tree necessitates searches in multiple sorted string tables during read operations, resulting in high disk I/O overhead and increased query times, which decreases the query efficiency of database systems. To address this challenge, we propose a machine learning-based bloom filter adjustment mechanism to enhance the read performance of LSM-tree. Specifically, we introduce an asynchronous bloom filter memory estimation algorithm and propose a dynamic resource adjustment method through a defined multi-container data structure and bloom filter configuration unit configuration protocol. Experimental results demonstrate that our proposed system, i.e., RallyDB, reduces read latency by 25% compared to ElasticBF, which significantly improves database performance.