How confident are students in their misconceptions about hypothesis tests?

Both researchers and teachers of statistics have made considerable efforts during the last decades to re-conceptualize statistics courses in accordance with the general reform movement in mathematics education. However, students still hold misconceptions about statistical inference even after following a reformed course. The study presented in this paper addresses the need to further investigate misconceptions about hypothesis tests by (1) documenting which misconceptions are the most common among university students of introductory courses of statistics, and (2) concentrating on an aspect of research about misconceptions that has not yet received much attention thus far, namely the confidence that students have in their misconceptions. Data from 144 college students were collected by means of a questionnaire addressing the most common misconceptions found in the literature about the definitions of hypothesis test, p-value, and significance level. In this questionnaire, students were asked to select a level of confidence in their responses (from 0 to 10) for each item. A considerable number of participants seemed to hold misconceptions and lower levels of concept-specific self-perceived efficacy were found to be related to misconceptions more than to the correct answers. On average, students selected significantly lower levels of confidence for the question addressing the definition of the significance level than for the other two items. Suggestions for further research and practice that emerge from this study are proposed.