While significant strides have been made in Information Retrieval (IR) and Recommender Systems (RecSys), reproducibility and replicability of findings in these fields remain elusive, often hindered by methodological inconsistencies and pressures for high-impact results. Despite the introduction of reproducibility tracks, guidelines, and frameworks across top conferences, reproducibility standards remain inconsistently applied, and the motivation for rigorously reproducible research is limited in practice. This tutorial aims to address these challenges by providing a structured introduction to essential tools and methodologies that ensure reproducibility in IR and RecSys research. By focusing on both foundational concepts—such as selecting appropriate baselines and designing replicable experimental pipelines—and practical case studies demonstrating the impact of minor experimental variations, this session will equip researchers with actionable strategies for producing robust, transparent studies. This tutorial ultimately seeks to foster a culture of scientific integrity within the IR and RecSys communities, supporting more reliable and impactful research advancements.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Reproducibility and Replicability in Information Retrieval: A Path Towards Scientific Integrity and Effective Research

  • Antonio Ferrara,
  • Claudio Pomo,
  • Nicola Tonellotto

摘要

While significant strides have been made in Information Retrieval (IR) and Recommender Systems (RecSys), reproducibility and replicability of findings in these fields remain elusive, often hindered by methodological inconsistencies and pressures for high-impact results. Despite the introduction of reproducibility tracks, guidelines, and frameworks across top conferences, reproducibility standards remain inconsistently applied, and the motivation for rigorously reproducible research is limited in practice. This tutorial aims to address these challenges by providing a structured introduction to essential tools and methodologies that ensure reproducibility in IR and RecSys research. By focusing on both foundational concepts—such as selecting appropriate baselines and designing replicable experimental pipelines—and practical case studies demonstrating the impact of minor experimental variations, this session will equip researchers with actionable strategies for producing robust, transparent studies. This tutorial ultimately seeks to foster a culture of scientific integrity within the IR and RecSys communities, supporting more reliable and impactful research advancements.