Advancing OLAP on Hadoop: Serverless and Cloud-Native Approaches
摘要
Online Analytical Processing (OLAP) plays a crucial role in business intelligence by enabling complex analytical queries over large volumes of structured data. With the rapid expansion of big data, achieving high-performance OLAP on distributed clusters of commodity hardware has emerged as a significant research focus. This paper presents a comprehensive performance evaluation of contemporary SQL-on-Hadoop systems—Apache Hive with Tez, Apache Spark SQL, and Cloudera Impala—utilizing the TPC-H benchmark across dataset sizes ranging from 100 GB to 1 TB. Experimental findings indicate that Spark SQL consistently outperforms Hive and Impala in complex multi-join queries, primarily due to its Catalyst optimizer and Tungsten execution engine. In contrast, Impala demonstrates lower latency in simple aggregation workloads. Scalability assessments highlight Spark SQL’s superior performance as node count and data volume increase, while Hive shows resilience under constrained memory environments, and Impala suffers performance degradation at scale. Additionally, comparisons between physical and virtual clusters reveal that physical deployments offer better performance, particularly for I/O-intensive queries. The study further explores how emerging trends such as cloud-native architectures, containerization, and serverless frameworks influence OLAP scalability, fault tolerance, and cost-effectiveness. These insights are intended to guide the selection and tuning of SQL-on-Hadoop systems for real-time, large-scale analytical processing.