A simulation comparison of parametric and nonparametric dimensionality detection procedures

Recently, nonparametric methods have been proposed that provide a dimensionally based description of test structure for tests with dichotomous items. Because such methods are based on different notions of dimensionality than are assumed when using a psychometric model, it remains unclear whether these procedures might lead to a different understanding of test structure. In this study, the nonparametric procedures DETECT (Kim, 1994; Zhang & Stout, 1999a) and the Mokken Scaling Program (MSP; Molenaar & Sijtsma, 2000) are compared against a procedure that groups items based on their estimated discrimination parameters, here referred to as parametric cluster analysis (Miller & Hirsch, 1992). The three procedures are compared by simulation across a range of conditions, including conditions in which the generating model differs from that assumed by the parametric model. Results suggest that although all methods tend to be similarly affected by the studied simulation conditions, they are differentially sensitive to the correlations between latent dimensions: DETECT performs similarly to parametric cluster analysis, whereas MSP requires much lower correlations before identifying distinct dimensions. Also, the parametric method appears robust against model misspecification.