The practice of grading candidates according to their performance is widespread and of utmost importance, where the key is to assess each candidate’s performance with reliable information. This paper aims to investigate peer grading in scenarios with ground truths, where there is an unknown ground truth represented by a real number for each candidate’s performance. The final grade assigned to each candidate is determined by the assessments, which are assumed as unbiased estimators of the candidate’s ground truth, made by both the experts and the candidates. We are interested in peer grading mechanisms that not only provide unbiased grades but are also resistant to strategic behaviors such as unilateral strategy deviation and collusion, which poses a challenging task in peer mechanism design. Generally, we make two contributions to the study of peer grading. First, we establish the impossibility of designing non-trivial grading mechanisms when the number of candidates is less than three. Second, we propose a practical and non-trivial grading mechanism, called the mean externality mechanism, for settings with at least three candidates, which is proved to be unbiased, strategy-proof and collusion-proof.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Strategy-Proof and Collusion-Proof Peer Grading Mechanism

  • Bin Li,
  • Xiaoyu Du

摘要

The practice of grading candidates according to their performance is widespread and of utmost importance, where the key is to assess each candidate’s performance with reliable information. This paper aims to investigate peer grading in scenarios with ground truths, where there is an unknown ground truth represented by a real number for each candidate’s performance. The final grade assigned to each candidate is determined by the assessments, which are assumed as unbiased estimators of the candidate’s ground truth, made by both the experts and the candidates. We are interested in peer grading mechanisms that not only provide unbiased grades but are also resistant to strategic behaviors such as unilateral strategy deviation and collusion, which poses a challenging task in peer mechanism design. Generally, we make two contributions to the study of peer grading. First, we establish the impossibility of designing non-trivial grading mechanisms when the number of candidates is less than three. Second, we propose a practical and non-trivial grading mechanism, called the mean externality mechanism, for settings with at least three candidates, which is proved to be unbiased, strategy-proof and collusion-proof.