Ensuring reliability in microservice architectures requires sophisticated fault injection methods to test system resilience. Traditional approaches to fault injection either rely on random failure generation, which can overlook critical faults, or involve complex modifications to low-level interfaces, risking further instability. This paper introduces Request-Level Fault Injection (RLFI), a novel approach leveraging the baggage annotation capability of the Opentracing framework to propagate failure flags at the request level without altering low-level protocols. By integrating RLFI with tracing, we gain comprehensive insights into system behavior under fault conditions, allowing developers to diagnose and address failure points effectively. RLFI supports fine-grained control over fault scenarios by combining decorator-based injection with Lineage Driven Fault Injection (LDFI), offering a robust framework for debugging distributed systems. Our empirical results show that RLFI enhances fault detection rates and reduces system impact compared to traditional methods, enabling efficient and precise testing in complex architectures. This work represents a significant advance in fault tolerance for microservices, presenting a practical, low-overhead method to ensure service reliability and availability.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Distributed System Reliability Through Request-Level Fault Injection and Fine-Grained Tracing

  • Ranjith Kumar Ramakrishnan,
  • Mahendra Sadineni,
  • Jai Jaswant Lekkala

摘要

Ensuring reliability in microservice architectures requires sophisticated fault injection methods to test system resilience. Traditional approaches to fault injection either rely on random failure generation, which can overlook critical faults, or involve complex modifications to low-level interfaces, risking further instability. This paper introduces Request-Level Fault Injection (RLFI), a novel approach leveraging the baggage annotation capability of the Opentracing framework to propagate failure flags at the request level without altering low-level protocols. By integrating RLFI with tracing, we gain comprehensive insights into system behavior under fault conditions, allowing developers to diagnose and address failure points effectively. RLFI supports fine-grained control over fault scenarios by combining decorator-based injection with Lineage Driven Fault Injection (LDFI), offering a robust framework for debugging distributed systems. Our empirical results show that RLFI enhances fault detection rates and reduces system impact compared to traditional methods, enabling efficient and precise testing in complex architectures. This work represents a significant advance in fault tolerance for microservices, presenting a practical, low-overhead method to ensure service reliability and availability.