Data Replica Placement Approach in Scientific Cloud Applications
摘要
With the rapid development of cloud computing, an increasing number of scientific teams are leveraging cloud platforms to process complex tasks. In such environments, data centers consist of multiple nodes, each storing subsets of datasets required for task execution. When a task is scheduled to a specific node, and the necessary datasets are not fully available locally, the missing data must be transferred from other nodes. This inevitable data transmission not only consumes substantial network bandwidth but also degrades task execution efficiency. To bridge this gap, we propose a data replica placement approach (DRPA) that combines clustering and genetic algorithms, and further optimize it with a replica deactivation principle. This approach reduces inter-node data transfers and improves task efficiency. We conduct a series of experiments comparing DRPA with the Firefly Algorithm-Based Placement (FABP) strategy, the Lyapunov-Based Placement (LBP) strategy, and the Hadoop Distributed File System (HDFS). The experimental results unveil that DRPA significantly reduces the total data transmission volume, with a reduction ranging from 50.3% to 79.9% .