Scholarlink

<p>The explosive growth of data in modern organizations has led to a greater reliance on distributed storage systems such as Hadoop Distributed File System (HDFS). To ensure reliability, HDFS traditionally keeps multiple replicas of each data block supported by a static placement strategy that distributes replicas across racks to balance availability and bandwidth usage. While effective for fault tolerance, this redundancy comes at a high storage and maintenance cost. However, existing surveys either treat Hadoop/HDFS redundancy as one component within broader cloud replication taxonomies, or focus on a limited subset of HDFS mechanisms. As a result, a systematic synthesis that jointly covers replication factor control, replica placement, and erasure coding in Hadoop/HDFS remains limited. This paper presents a systematic review of recent strategies aimed at making data redundancy in Hadoop smarter and more efficient. The review adopts a new taxonomy into three categories namely <b>Erasure coding</b> methods, <b>Reactive replication</b> strategies, and <b>Predictive replication</b> strategies. The selected papers are analyzed according to several criteria. Overall, the survey reveals a clear trend: Hadoop’s redundancy management is evolving beyond fixed triple replication and static placement toward more adaptive schemes that consider data popularity, workload dynamics, and cost-performance trade-offs. Finally, we discuss open challenges and outline how addressing these gaps can lead to more resilient and efficient big data storage in the future.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Toward smarter redundancy in Hadoop: a systematic review of replication and erasure coding approaches

  • 期刊论文

  • 发表日期：

    2026年7月3日

查看全文
  • Helmi Tlich,
  • Hanène Chettaoui,
  • Tarek Hamrouni

摘要

The explosive growth of data in modern organizations has led to a greater reliance on distributed storage systems such as Hadoop Distributed File System (HDFS). To ensure reliability, HDFS traditionally keeps multiple replicas of each data block supported by a static placement strategy that distributes replicas across racks to balance availability and bandwidth usage. While effective for fault tolerance, this redundancy comes at a high storage and maintenance cost. However, existing surveys either treat Hadoop/HDFS redundancy as one component within broader cloud replication taxonomies, or focus on a limited subset of HDFS mechanisms. As a result, a systematic synthesis that jointly covers replication factor control, replica placement, and erasure coding in Hadoop/HDFS remains limited. This paper presents a systematic review of recent strategies aimed at making data redundancy in Hadoop smarter and more efficient. The review adopts a new taxonomy into three categories namely Erasure coding methods, Reactive replication strategies, and Predictive replication strategies. The selected papers are analyzed according to several criteria. Overall, the survey reveals a clear trend: Hadoop’s redundancy management is evolving beyond fixed triple replication and static placement toward more adaptive schemes that consider data popularity, workload dynamics, and cost-performance trade-offs. Finally, we discuss open challenges and outline how addressing these gaps can lead to more resilient and efficient big data storage in the future.