Safety-critical systems that integrate machine learning (ML) components have gained significant traction in recent years. However, current safety analysis methods lack a systematic approach to address the uncertainty inherent in ML system decision-making. Consequently, performing thorough risk analysis for ML systems with these methods is often time-consuming and makes it difficult to identify risks associated with uncertainty. In this study, we introduce a novel risk analysis method called eAI-Risk, designed for complex ML systems comprising multiple ML components. This method employs a data-driven and scenario-based approach, providing a structured framework for systematically identifying factors that contribute to high risks in ML systems and for evaluating potential countermeasures. A key feature of eAI-Risk is its capacity to identify risks within ML systems based on the classification patterns of the ML components and the action patterns defined by the system, while considering the uncertainty that characterizes ML systems. In addition, risks are quantitatively assessed from two perspectives: severity and likelihood. The product of these values generates a risk score, facilitating the prioritization of risks to address. This study illustrates the effectiveness of eAI-Risk by applying it to an automated driving system. Through the eAI-Risk process cycle, we anticipate that optimal risk mitigation strategies can be effectively identified and implemented.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Data-Driven and Scenario-Based Risk Analysis Considering Uncertainty in Machine Learning Systems

  • Toshiya Okubo,
  • Hironori Washizaki,
  • Naoyasu Ubayashi,
  • Nobukazu Yoshioka,
  • Hironori Takeuchi,
  • Truong Vinh Truong Duy

摘要

Safety-critical systems that integrate machine learning (ML) components have gained significant traction in recent years. However, current safety analysis methods lack a systematic approach to address the uncertainty inherent in ML system decision-making. Consequently, performing thorough risk analysis for ML systems with these methods is often time-consuming and makes it difficult to identify risks associated with uncertainty. In this study, we introduce a novel risk analysis method called eAI-Risk, designed for complex ML systems comprising multiple ML components. This method employs a data-driven and scenario-based approach, providing a structured framework for systematically identifying factors that contribute to high risks in ML systems and for evaluating potential countermeasures. A key feature of eAI-Risk is its capacity to identify risks within ML systems based on the classification patterns of the ML components and the action patterns defined by the system, while considering the uncertainty that characterizes ML systems. In addition, risks are quantitatively assessed from two perspectives: severity and likelihood. The product of these values generates a risk score, facilitating the prioritization of risks to address. This study illustrates the effectiveness of eAI-Risk by applying it to an automated driving system. Through the eAI-Risk process cycle, we anticipate that optimal risk mitigation strategies can be effectively identified and implemented.