Data contamination occurs when test instances have been compromised during a training stage of building a machine learning model. The consequences of this phenomenon over the quality of learning data are crucial when evaluating a learned predictor, since it could distort the assessment of the actual capabilities of the system. Its study has recently gained more traction in the research on Large Language Models, where it is common to chase performances in order to support claims about model abilities. Since the field of Information Retrieval increasingly studies and develops approaches that rely on these data-centric technologies, this position paper considers the phenomenon of data contamination in terms of its possible consequences for this field.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

On the Implications of Data Contamination for Information Retrieval Systems

  • Darío Garigliotti

摘要

Data contamination occurs when test instances have been compromised during a training stage of building a machine learning model. The consequences of this phenomenon over the quality of learning data are crucial when evaluating a learned predictor, since it could distort the assessment of the actual capabilities of the system. Its study has recently gained more traction in the research on Large Language Models, where it is common to chase performances in order to support claims about model abilities. Since the field of Information Retrieval increasingly studies and develops approaches that rely on these data-centric technologies, this position paper considers the phenomenon of data contamination in terms of its possible consequences for this field.