Monte Carlo evaluation of two-level logistic regression for assessing person fit

Person fit is the degree to which an item response model fits for individual examinees. Reise (2000) described how two-level logistic regression can be used to detect heterogeneity in person fit, evaluate potential predictors of person fit heterogeneity, and identify potentially aberrant individuals. The method has apparently never been applied to empirical data or evaluated in a simulation study. The present research applies Reise's method to empirical data obtained from university undergraduates measured on the Fear of Negative Evaluation scale. Additionally, Reise's method is evaluated under conditions varying according to the type of aberrancy, level of test reliability, and scale length. Statistical power to detect aberrancy was highly dependent on manipulated variables, and some results were affected by bias in model parameters that was due to the aberrant responders. Nevertheless, Reise's method generally performed well and detected aberrant individuals either as well as, or better than, the well-established l(z) person-fit statistic.