Validation of automated scoring of science assessments

被引：93

作者：

Liu, Ou Lydia ^{[1
]}

Rios, Joseph A. ^{[1
]}

Heilman, Michael ^{[1
]}

Gerard, Libby ^{[2
]}

Linn, Marcia C. ^{[2
]}

机构：

[1] Educ Testing Serv, Princeton, NJ 08541 USA

[2] Univ Calif Berkeley, Grad Sch Educ, Berkeley, CA 94720 USA

来源：

JOURNAL OF RESEARCH IN SCIENCE TEACHING | 2016年 / 53卷 / 02期

基金：

美国国家科学基金会;

关键词：

automated scoring; c-rater-ML; science assessment; KNOWLEDGE INTEGRATION; WEIGHTED KAPPA; AGREEMENT;

D O I：

10.1002/tea.21299

中图分类号：

G40 [教育学];

学科分类号：

040101 ; 120403 ;

摘要：

Constructed response items can both measure the coherence of student ideas and serve as reflective experiences to strengthen instruction. We report on new automated scoring technologies that can reduce the cost and complexity of scoring constructed-response items. This study explored the accuracy of c-rater-ML, an automated scoring engine developed by Educational Testing Service, for scoring eight science inquiry items that require students to use evidence to explain complex phenomena. Automated scoring showed satisfactory agreement with human scoring for all test takers as well as specific subgroups. These findings suggest that c-rater-ML offers a promising solution to scoring constructed-response science items and has the potential to increase the use of these items in both instruction and assessment. (c) 2015 Wiley Periodicals, Inc. J Res Sci Teach 53: 215-233, 2016.

引用

页码：215 / 233

页数：19

共 45 条

[11]

Cohen J, 2013, Statistical power analysis for the behavioral sciences, DOI [10.4324/9780203771587, DOI 10.4324/9780203771587]

[12]

Dzikovska M.O., 2013, P 7 INT WORKSHOP SEM

[13]

Dzikovska MyroslavaO., 2010, P ACL 2010 SYST DEM, P13

[14] EQUIVALENCE OF WEIGHTED KAPPA AND INTRACLASS CORRELATION COEFFICIENT AS MEASURES OF RELIABILITY [J].