Development of the social metacognition inventory for online collaborative argumentation: construct validity and reliability

At present, with the rapid development of the internet and the gradual promotion of online collaborative learning, the social regulation of learning is receiving increasing attention, which involves socially shared metacognition, one facet of social metacognition. To date, social regulation of learning or socially shared metacognition have been widely studied using qualitative approaches. Although a variety of scales have been developed to measure metacognition in traditional individual learning, little work has been done to develop a scale to measure social metacognition in collaborative learning contexts. This study originally developed a social metacognition inventory consisting of 24 indicators by referring to the literature for assessing beliefs of other persons (BOP), awareness of other persons’ thinking (AOPT), judgment of other persons’ emotions (JOPE), co-regulation of each other’s thinking (CREOT), and evaluation of other persons’ thinking (EOPT). After EFA using 218 undergraduates’ questionnaires of social metacognition in collaborative argumentation on a social psychological issue from a Sino-Foreign Cooperative Educational Institution, 17 indicators showed good factorability and reliability. After CFA using another 300 questionnaires on social metacognition in collaborative argumentation about the pandemic received from undergraduates who come from 52 countries in the International College of Education, among the 17 indicators derived from the first sample’s EFA, three indicators had high correlation with others. Finally, based on the reviews of three experts, these three indicators were deleted. The remaining 14 indicators formed good construct validity with acceptable convergent and discriminant validities. In addition, the multi-group invariance test demonstrated that the structural model of the Social Metacognition Inventory has better configural invariance, which indicates that it can be generalized to other online collaborative argumentation contexts. The Social Metacognition Inventory can be used to quantify social metacognition in online collaborative argumentation when administering a large-scale experiment.