Side effects and the enactment of accountability: results of a comparative study in two German federal states

The paper proposes to study side effects of accountability in education within the theoretical framework of enactment research. The potential value of this approach for the study of side effects is shown by using the example of the side effect "Dependency on Expert Judgements." Therefore, findings from the research project "Unintended Effects of Accountability in the School System" (acronym "Nefo") are presented. The project entailed an analysis of policy documents to describe the accountability contexts under study, as well as a survey study with 2637 participating teachers and principals to examine the distribution of side effects in the no- and low-stakes contexts of the four German federal states of Berlin, Brandenburg, Thuringia, and Rhineland-Palatinate. The findings for Berlin and Thuringia are triangulated with the results from a qualitative in-depth group-discussion substudy of the Nefo project on how teachers in Berlin and Thuringia deal with accountability measures. The analysis of policy documents reveals that in Berlin, performance results gain the status of objective data that teachers need to improve their work, while teachers in Thuringia are prompted to judge by themselves the meaning of performance results for their teaching. This suggests that "Dependency on Expert Judgements" is a larger issue in Berlin than in Thuringia. However, contrary to what would be expected, the findings of the survey study indicate that the side effect plays a greater role in Thuringia than in Berlin. To explain this counterintuitive finding, teachers' responses to standards-based accountability in Berlin and Thuringia are delineated. The paper shows that the study of meaning-making processes, which is central to the enactment framework, must not only be ignored if one tries to understand the processes that lead to the occurrence or absence of side effects. It is also important to prevent fallacies in the interpretation of survey data.