Attribute-Level and Pattern-Level Classification Consistency and Accuracy Indices for Cognitive Diagnostic Assessment

Classification consistency and accuracy are viewed as important indicators for evaluating the reliability and validity of classification results in cognitive diagnostic assessment (CDA). Pattern-level classification consistency and accuracy indices were introduced by Cui, Gierl, and Chang. However, the indices at the attribute level have not yet been constructed. This study puts forward a simple approach to estimating the indices at both the attribute and the pattern level through one single test administration. Detailed elaboration is made on how the upper and lower bounds for the attribute-level accuracy can be derived from the variance of error of the attribute mastery probability estimate. In addition, based on Cui's pattern-level indices, an alternative approach to estimating the attribute-level indices is also proposed. Comparative analysis of simulation results indicate that the new indices are very desirable for evaluating test-retest consistency and correct classification rate.