Using student feedback on examination questions to promote fairness, item validity, and learning

Ten years' experience with a system of modifying the scoring of examinations in pharmacology based on feedback comments from students on individual questions are analyzed. Each examination (3 per year for 10 years), approximately 100 items in length, was subjected to rigorous editing by a faculty committee and an independent reviewer. Putatively correct answers were posted after She examination, and students submitted comments anonymously on questions they thought justified alternative answers. Those comments accepted as valid by faculty were used to modify final scoring. A mean of 7.17 +/- 0.85 questions per examination (+/- SE) had valid alternative answers (range, 2-17 questions; median, 6-7 questions; n = 30 examinations). No examination escaped with fewer than two items that had alternative answers accepted in response to students' comments. The mean number of deleted items was 1.63 +/- 0.34 (range, 0-6). This feedback comments system has received consistently high praise from students who say that it improves fairness and validity in examinations. It has now been adopted for use in objective examinations in the major courses in this school of medicine.