Measuring the Impact of Distractors on Student Learning Gains while Using Proof Blocks

Background: Proof Blocks is a software tool that enables students to construct proofs by assembling prewritten lines and gives them automated feedback. Prior work on learning gains from Proof Blocks has focused on comparing learning gains from Proof Blocks against other learning activities such as writing proofs or reading. Purpose: The study described in this paper aims to compare learning gains from variations of Proof Blocks. Specifically, we attempt to quantify the difference in learning gains for students who complete Proof Blocks problems with and without distractor lines. Methods: We conducted a randomized controlled trial with three experimental groups: a control group that completed an off-topic Proof Blocks activity, one that completed a Proof Blocks activity without distractor lines, and one that completed a Proof Blocks activity with distractor lines. All three groups read a book chapter on proof by induction before completing their activity. Findings: The group that completed the Proof Blocks activity with distractors performed better on the posttest than the group that completed the Proof Blocks without distractors, who in turn performed better than the group that completed the off-topic Proof Blocks activity. However, none of these differences were statistically significant. While the results are consistent with the idea that distractors could be beneficial for learning, future work will need to confirm.