Many real-world machine learning (ML) projects still fail. One reason is that the results do not meet the stakeholders’ implicit needs and expectations. In manufacturing use cases, e.g., those goals are often related to cost reduction through increased automation or reduced scrap rates. Since different misclassifications can result in varying consequences and costs, decision preferences should be considered during the ML process. We present the results of a systematic mapping study. The primary objective of this study was to identify existing methods for integrating the diverse decision preferences of stakeholders, whether monetary or non-monetary, into the ML process in a goal-oriented manner: We found twelve case studies and nine methodological approaches that pursue this goal. From this result, five classes of methods were identified to help practitioners find an appropriate approach to their problem and to point researchers to different research directions. We also derive a taxonomy of the different drivers of ML decision preferences.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Goal-Orientation in Machine Learning Development: A Systematic Mapping Study

  • Kristina Dachtler,
  • Alexander Schiendorfer

摘要

Many real-world machine learning (ML) projects still fail. One reason is that the results do not meet the stakeholders’ implicit needs and expectations. In manufacturing use cases, e.g., those goals are often related to cost reduction through increased automation or reduced scrap rates. Since different misclassifications can result in varying consequences and costs, decision preferences should be considered during the ML process. We present the results of a systematic mapping study. The primary objective of this study was to identify existing methods for integrating the diverse decision preferences of stakeholders, whether monetary or non-monetary, into the ML process in a goal-oriented manner: We found twelve case studies and nine methodological approaches that pursue this goal. From this result, five classes of methods were identified to help practitioners find an appropriate approach to their problem and to point researchers to different research directions. We also derive a taxonomy of the different drivers of ML decision preferences.