Technological indicators of impact of course management systems

Over the last decade, many course and learning management systems have been developed and deployed in educational environments. Many more new paradigms forlearning will become available as further progress is made in the underlying technologies. Therefore, it is important to develop an evaluation strategy to measure the effectiveness of such systems with regard to delivery, learning, and assessment. The goal of this study was to provide the first step in developing such a strategy. A set of technological indicators was developed to determine how, and when the Blackboard learning-management system was used in a university environment. The pattern of use of the system was derived by examining a set of logs generated by the Web server, the Blackboard system, and the operating system. Our results indicated that during the course of a semester almost 70% of the students and 20% of the instructors used the system. While a significant amount of course material was made available by the instructors, the system seemed to be used primarily as a publishing environment. With these results as background information, we suggest that researchers next investigate the interesting questions of how well the system works for learning and assessment.