Assessing syntactic sophistication in L2 writing: A usage-based approach

Over the past 45 years, the construct of syntactic sophistication has been assessed in L2 writing using what Bulte and Housen (2012) refer to as absolute complexity (Lu, 2011; Ortega, 2003; Wolfe-Quintero, Inagaki, & Kim, 1998). However, it has been argued that making inferences about learners based on absolute complexity indices (e.g., mean length of t-unit and mean length of clause) may be difficult, both from practical and theoretical perspectives (Norris & Ortega, 2009). Furthermore, indices of absolute complexity may not align with some prominent theories of language learning such as usage-based theories (e.g., Ellis, 2002a,b). This study introduces a corpus-based approach for measuring syntactic sophistication in L2 writing using a usage-based, frequency-driven perspective. Specifically, novel computational indices related to the frequency of verb argument constructions (VACs) and the strength of association between VACs and the verbs that fill them (i.e., verb-VAC combinations) are developed. These indices are then compared against traditional indices of syntactic complexity (e.g., mean length of T-unit and mean length of clause) with regard to their ability to model one aspect of holistic scores of writing quality in Test of English as a Foreign Language (TOEFL) independent essays. Indices related to usage-based theories of syntactic development explained greater variance (R-2=.142) in holistic scores of writing quality than traditional methods of assessing syntactic complexity (R-2=.058). The results have important implications for modeling syntactic sophistication, L2 writing assessment, and AES systems.