Preparing and Analyzing Log and Process Data in Large-Scale Assessments
摘要
Over the last decade, France, like many other countries, has been undergoing a transition from paper-based to digital assessments in order to measure student performance in education. There is a rising interest in France in technology-enhanced items, which offer new ways to assess traditional competencies, as well as address higher-order skills and the so-called twenty-first century skills. The rich data captured by these items, often referred to as process data, allow insight into how students tackle the item and their response strategies. “Big data” solutions were set up in France in order to handle the large volume and complexity of these process data from technology-based assessments. Data-driven approaches, stemming from the domain of educational data mining, as well as theory-driven approaches, drawing on expert knowledge of the field, were applied to the process data. Our experience has taught us that making process data meaningful necessitates integrating three paramount aspects of our work: didactics, technology, and analytics. Coordinating findings, constraints, and needs of these three aspects is essential to devise process indicators, draw hypotheses, and allow for valid interpretation. Valid process data interpretation enables findings to be actionable in the field, from an “assessment for learning” perspective.