Understanding the Effects of Intervention on Computer Science Student Behaviour in On-line Forums

A key challenge for educators using online discussion forums is how to provide effective feedback and support to students, to ensure they are engaged with discussions on the forums, and do not disengage from the course. In addition, there is a significant problem of scale to address. Even relatively low population forums of less than a hundred students can generate thousands of posts, so it is infeasible for the lecturer to monitor every discussion to identify disengaging students. There is a need to understand the act of intervention, in order to provide automated tools to better assist teaching staff with this task. Measuring the impact of intervention can be challenging and requires us to, first, understand what "standard" behaviour looks like across different student groups and identify topics where intervention would be most effective. In this paper, we identify the impact of intervention on different groups of students, characterising their behaviour in terms of response time and activity, compare the different responses to programming-related questions and other questions, and identify the useful aspects of this study for computer science educators. We conduct an initial examination of the impact of the nature of the question on intervention effectiveness and propose an analysis method that can be applied to any computer science forum. we showcase the application of the method to three courses. Our results indicate that associating student activity with the number of forum posts is misleading, as students who are only reading the forums respond also to intervention.