Scholarlink

Accuracy estimation is crucial for reliable real-world machine learning applications, particularly when the target labels are observed with delay. This paper proposes a novel guided binary cross-entropy (GBCE) loss designed explicitly for training a “check model,” i.e., a model dedicated to checking predictions to estimate accuracy. The GBCE loss focuses on learning prediction probabilities for predicted labels, which is critical for accuracy estimation. Compared to the cross-entropy (CE) loss, which is commonly employed in related methods, the GBCE loss directly minimizes the upper bound of the accuracy estimation error. We theoretically analyze our method, providing its generalization error bound. Our numerical experiments further validate our GBCE-based accuracy estimation, demonstrating its superior performance by reducing accuracy estimation error by up to 30.2%.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

GBCE: Enhanced Training Loss to Estimate Accuracy of Models in Production

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Ryuta Matsuno

摘要

Accuracy estimation is crucial for reliable real-world machine learning applications, particularly when the target labels are observed with delay. This paper proposes a novel guided binary cross-entropy (GBCE) loss designed explicitly for training a “check model,” i.e., a model dedicated to checking predictions to estimate accuracy. The GBCE loss focuses on learning prediction probabilities for predicted labels, which is critical for accuracy estimation. Compared to the cross-entropy (CE) loss, which is commonly employed in related methods, the GBCE loss directly minimizes the upper bound of the accuracy estimation error. We theoretically analyze our method, providing its generalization error bound. Our numerical experiments further validate our GBCE-based accuracy estimation, demonstrating its superior performance by reducing accuracy estimation error by up to 30.2%.