Examining the validity of the Arizona English Language Learners Assessment cut scores

The Arizona English Language Learners Assessment (AZELLA) is used by the Arizona Department of Education to determine which children should receive English support services. AZELLA results are used to determine if children are either proficient in English or have English language skills in one of four non-proficient categories (pre-emergent, emergent, basic, intermediate). Children who test at or above the proficient cut score in English are placed in mainstream classes without English language support. Children who obtain scores below the proficient cut scores receive English language support services in state-mandated Structured English Immersion classes. Whenever tests are used to make high-stakes decisions, especially about vulnerable populations (e.g., children), it is the test developers' responsibility to ensure the instrument yields fair and valid results. When cut scores are used as the primary interpretation of the test they are key to establishing the test's validity. This validation study found that cut scores for the AZELLA are of questionable validity. The procedure used to set the cut scores is criticized by national measurement experts as ineffective and obsolete. Further, the test developers do not adequately establish the expertise of the judges used to set the cut scores. Evidence from the cut-score-setting process indicates judges did not come to consensus at the kindergarten level. Analysis of empirical evidence suggests cut scores over-identify kindergarten children and under-identify older children. Finally, the test developers rejected 85% of the cut scores recommended by the standard-setting judges, setting cut scores higher than recommended for kindergarten and lower than recommended for older children, without describing their process or rationale.