Alternatives in evaluating multimedia in secondary school science teaching

While different approaches to evaluation will yield different results, depending on the purpose of the evaluation this article describes an evaluation approach that was aimed at investigation the mental models of users of the programme. The study was driven by questions about the differences in mental models of the instructional designers and the learners, the time learners spent working through the program and the observable changes in their mental models. In this design experiment, a program was developed to teach basic principles of electricity. Three boys and three girls one each of high, medium and low achievement in science were selected from an advantaged urban school, and a similar sample was taken from a disadvantaged rural school. They were asked to draw their impressions of various concepts of electricity and then allowed free access to the program, where they could visit any section even if they had not completed a previous one. Afterwards they were asked to draw sketches again. Other instruments included an opinion questionnaire and observation of the learners working with a "think aloud" protocol. It was found that there were considerable differences in the mental models of the learners and designers about what to expect from computer-based learning. While navigational freedom allowed fast learners to move through work that they knew already, weaker learners tended to get lost. The sketches that learners made before and after exposure to the program provided valuable insights into the growth of their understanding of the concepts. (C) 2007 Elsevier Ltd. All rights reserved.