Development, validation and evaluation of a multiple-choice questionnaire to identify and to characterize preservice teachers' science views

In recent years, there has been an obvious effort among researchers in science education about the epistemological conceptions of future teachers. This work fits into this line of research, and shows the design, development and validation of a multiple-choice questionnaire to evaluate scientific beliefs of secondary future teachers (COMVDC). The main feature of the design of the questionnaire is that we have tried to fit it to the content on the nature of science that we should teach to future teachers of secondary schools. This is, after doing a literature review on the epistemological conceptions of teachers, we conclude the desirability of a new questionnaire that could be done in a hour, in the context of teacher training at secondary school. So, a pre-post-test administration may reveal the degree of learning related to the nature of science. Specifically, this study aims to: a) Show the contents and topics about the construction and nature of science that might be of interest for future teachers of secondary schools, and that, accordingly, we are trying to evaluate with the questionnaire. b) Describe the process for the development of the multiple-choice items. c) Show in detail the process used in the validation of the initial questionnaire after its administration to a sample of 45 high school teachers in a CAP course developed in 2003. d) Discuss the strengths and weaknesses of the final questionnaire, after administering to a second sample of 50 high school teachers in a CAP course developed in 2005. The main goal in 2003 was the purification and validation of items, but in 2005, the special emphasis was on the usefulness of the questionnaire for the assessment of the teaching experience. In 2005, therefore, the questionnaire is used as the pretest and post-test. Another important difference between the two years is that the criteria for analyzing the suitability of each item in 2005 were more accurate, quantitative and consensus than those adopted in 2003. e) Draw conclusions about the results of the questionnaire and its potential to be introduced as a tool for both evaluation and instruction in the training of future science teachers. Specifically, to analyze the validity of the questionnaire, in 2003, sixteen indicators were calculated by which the improvement of the items of the questionnaire was possible. In 2005, twenty-two indicators were calculated, which enabled the estimation of 6 factors. The valuation of the goodness of each item (upper, middle and lower) is achieved by adding the values of each factor. This last process, in order to increase its objectivity, was done by 4 groups of prospective secondary teachers and 2 experts. In the paper, we show the questionnaire administered in 2005. It consists of 45 multiple choice items, of which thirteen items are low quality. Therefore, it would be more correct to exclude them to possible further analysis, for example, to calculate the differences between pretest and post-test. This analysis should be undertaken only with items of upper and middle quality, which have been more valid and discriminatory. The feasibility study is done through the correlations between the indicators in 2003 and 2005, and the correlations between the indicators of pretest and post-2005. These correlations indicate the degree of stability or reliability of the items to give the same results to samples or to different moments of learning. In this paper, we show that all the correlations between rates similar comment a few exceptions that are significant in at least the 95% of the cases. In general, the COMVDC and the self-applied mechanics make up a system that is converging toward a growing questionnaire that is valid and reliable. The COMVDC has very good qualities to be applied in educational contexts, such as the ability to be administered within an hour and be integrated into the teaching process as an effective teaching tool. Specifically, the results of COMVDC on each student, allow the identification of not only the progress of learning (when comparing results between pretest and post) but also, as the options of some items structured as epistemological tendencies can be done, for example, groups of students or constructivist tendency to follow trends of individual epistemological.