Multivariate reliability- and generalizability theory in teaching evaluation research

The evaluation of academic teaching takes place under varying conditions (timepoints, teachers, types of courses,...) using evaluation inventories. The validation criteria of measures are supposed to be generally valide. The generalizability theory (GT) and further the multivariate reliability and validation theory (MRVT) of Wittmann (1985) makes this implicit assumptions of constancy of conditions to a empirical testable hypothesis. A project for a evaluation of the new curriculum "Forest Science" 1996 in Freiburg serves as a database: 59 students of the 2. semester, 38 students of the 8. semester rated repeatedly time-sequential block courses on the "Heidelberg Inventory of teaching evaluation"(HILVE). High multivariate total reliability and generalizability of the HILVE-inventory can be shown. But there is a high variability of student judgments within and between courses. However, the HILVE is proven as change sensitive for the objective of teaching evaluation.