Overview of ELOQUENT 2025: Shared Tasks for Evaluating Generative Language Model Quality
摘要
ELOQUENT is a CLEF lab for evaluating generative language model quality with a focus on such aspects of quality that do not come to the fore with current standard test suites and test collections and to develop and promote new test regimes and methods that fit a multilingual application scenario for generative artificial intelligence. This year is the second year of ELOQUENT. This year’s experiment tracks have evolved from the first year: this year we continue challenging the capability of classifiers to distinguish machine-generated from human-authored text; we explore how consistent language models are in responding to value-oriented questions across languages and system settings; we test how accurately language models are able to predict human preferences between variants of generated material; and we investigate how well language models are able provide sensible topical quizzes to fit given target texts.