SAFE: Probabilistic Framework to Characterize Honest Reviewers
摘要
Computer Science research is mired in a long tradition of conference publications because of the opportunities a conference offers for quick dissemination of research findings, and describing research before peers in presentations. However, with too many papers vying for limited slots in the conference proceedings, maintaining hygiene in the review process poses a challenge for organizers. We propose to identify Safe And Focused Experts as reviewers from the technical program committee of a conference, based on their past research experience and bidding patterns. We match the similarities between the manuscripts and the reviewers’ publication, both at lexical and semantic levels, and score the extent to which the bid values align with the manuscripts. Based on the distribution of scores, we predict the probability of dubious bidding behavior of the reviewer using the Bayesian framework. The approach is data-driven and is motivated by the overarching goal of improving the quality and robustness of a conference review process. We validate the proposed framework by designing experiments for multiple threat scenarios and report encouraging results.