Scholarlink

<p>As open AI model repositories expand in scale and influence, scalable approaches to evaluate and govern widely adopted models are becoming increasingly essential. This study presents a predictive framework for model popularity, leveraging large-scale data from Hugging Face. Our analysis shows that model adoption is shaped by identifiable features, like task type, language coverage, creator identity, and geographic origin. Despite notable inconsistencies and gaps in model documentation, we achieved over 80% accuracy in predicting popularity, indicating that key adoption signals remain accessible. These findings underscore both the potential of popularity prediction as a triage mechanism for allocating evaluation resources and the critical need for stronger metadata standards to enhance model transparency and accountability. By positioning popularity prediction as a triage layer for downstream AI safety and governance efforts, our work provides practical insights for developers, researchers, and policymakers aiming to advance responsible oversight in open AI ecosystems. This is especially important as popular open AI models increasingly influence downstream applications and societal outcomes.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Towards Scalable Open AI Model Evals: Predicting Highly Adopted Open Models

  • 期刊论文

  • 发表日期：

    2026年7月18日

查看全文
  • Mana Azarm,
  • Esmat Sangari

摘要

As open AI model repositories expand in scale and influence, scalable approaches to evaluate and govern widely adopted models are becoming increasingly essential. This study presents a predictive framework for model popularity, leveraging large-scale data from Hugging Face. Our analysis shows that model adoption is shaped by identifiable features, like task type, language coverage, creator identity, and geographic origin. Despite notable inconsistencies and gaps in model documentation, we achieved over 80% accuracy in predicting popularity, indicating that key adoption signals remain accessible. These findings underscore both the potential of popularity prediction as a triage mechanism for allocating evaluation resources and the critical need for stronger metadata standards to enhance model transparency and accountability. By positioning popularity prediction as a triage layer for downstream AI safety and governance efforts, our work provides practical insights for developers, researchers, and policymakers aiming to advance responsible oversight in open AI ecosystems. This is especially important as popular open AI models increasingly influence downstream applications and societal outcomes.