A web application for aggregating conflicting reviewers' preferences

Drawing on social choice theory we derive a rationale in which each reviewer is asked to provide his or her second, third, and fourth choice in addition to his/her first choice recommendation regarding the acceptance/revision/rejection of a given manuscript. All reviewers' hierarchies of alternatives are collected and combined such that an overall ranking can be computed. Consequently, conflicting recommendations are resolved not by asking a third adjudicating reviewer for his/her recommendation as is usual editorial praxis in many scientific journals, but rather by using more information from the available judges. After a brief introduction into social choice theory and a description and justification of the maximum likelihood rule for ranking alternatives, we describe and demonstrate a public available web application that provides easy-to-use tools to apply these methods for aggregating conflicting reviewers' recommendations. This application might be accessed by editors to aid their decision process in case they receive conflicting recommendations by their reviewers.