A Novel Hybrid OptimizationAquila Flamingo SearchOptimizer and Genetic Algorithm for Efficient ETL Processes, Materialized View Selection, and Optimal Query Recommendation in Cloud-Based Data Warehousing
摘要
In Data Warehousing (DW) administration, Materialized View Selection (MVS) is essential for speeding up query processing. Data is usually stored in DWs as an accumulation of materialized views. The main challenge is deciding which views to materialize to meet response time targets at a lower cost. When building DWs, the Extraction, Transformation, and Loading (ETL) method is frequently employed, involving a substantial amount of information about the organization’s operations. MVS in DWs is a viable way to improve the analytical processing of massive amounts of historical information, assisting decision-making systems that employ data mining to find patterns in business. The need to access the source of the original data is reduced when details are materialized as views from various production databases within a DW, which increases query efficiency. A novel hybrid Aquila Flamingo Search Optimizer and Genetic Algorithm (AFSO-GA) is proposed to improve ETL procedures, MVS, and query suggestions. This method addresses issues with query execution, resource usage, and information integration efficiency in dynamic cloud systems. ETL procedures are enhanced by utilizing the AFSO-GA to speed up data movement and analysis. MVS is optimized using genetic programming, which improves query response times and data portability. To suggest the best queries, the algorithm uses workload analysis and previous query trends, enhancing system scalability and overall efficiency. This novel hybrid AFSO-GA approach is designed to meet the various demands of contemporary data-intensive applications while offering significant improvements in productivity and effectiveness within cloud-based DWs. The proposed AFSO-GA framework yielded results including a completion time of 226 s, a 6.32-minute view maintenance cost, and a 3.52-minute query processing cost. Outcomes demonstrate the approach’s effectiveness and affordability for selecting views in data warehousing configurations.