The widespread adoption of machine learning (ML) models in high-stakes decision-making underscores the need for rigorous trustworthiness assessments. Ensuring trustworthy AI requires evaluating multiple dimensions, including performance, fairness, and interpretability, to support reliable and ethical decision-making. This paper introduces a framework for systematically estimating the trustworthiness of ML models by aggregating diverse quantitative metrics into a unified trustworthiness score. The proposed design consists of modular components that assess key dimensions—performance, fairness, and interpretability—each contributing to an overall evaluation. A flexible weighting mechanism allows customization based on domain-specific priorities and regulatory requirements. The primary contribution of this work is a structured approach to integrating multiple trustworthiness dimensions into a single assessment framework. By offering a quantitative comparison of ML models, this framework enables researchers and practitioners to make informed decisions about model selection and deployment. While this framework remains in the design phase, future research will focus on implementation, empirical validation using real-world ML models, and refining metric selection strategies to enhance robustness. The broader implications of this work extend to regulatory compliance, model auditing, and trustworthiness benchmarking, fostering greater transparency and accountability in AI-driven systems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Assessing AI-Based System Acceptance Through the Design of a Trustworthiness Estimation Tool for Machine Learning Models

  • Jonathan Ugalde,
  • Rodrigo Salas,
  • Aurelio F. Bariviera,
  • María Paz Godoy

摘要

The widespread adoption of machine learning (ML) models in high-stakes decision-making underscores the need for rigorous trustworthiness assessments. Ensuring trustworthy AI requires evaluating multiple dimensions, including performance, fairness, and interpretability, to support reliable and ethical decision-making. This paper introduces a framework for systematically estimating the trustworthiness of ML models by aggregating diverse quantitative metrics into a unified trustworthiness score. The proposed design consists of modular components that assess key dimensions—performance, fairness, and interpretability—each contributing to an overall evaluation. A flexible weighting mechanism allows customization based on domain-specific priorities and regulatory requirements. The primary contribution of this work is a structured approach to integrating multiple trustworthiness dimensions into a single assessment framework. By offering a quantitative comparison of ML models, this framework enables researchers and practitioners to make informed decisions about model selection and deployment. While this framework remains in the design phase, future research will focus on implementation, empirical validation using real-world ML models, and refining metric selection strategies to enhance robustness. The broader implications of this work extend to regulatory compliance, model auditing, and trustworthiness benchmarking, fostering greater transparency and accountability in AI-driven systems.