Using BERT and XLNET for the Automatic Short Answer Grading Task

Over the last decade, there has been a considerable amount of research in automatic short answer grading (ASAG). The majority of previous experiments were based on a feature engineering approach and used manually-engineered statistical, lexical, grammatical and semantic features for ASAG. In this study, we aim for an approach that is free from manually-engineered features and propose an architecture for deep learning based on the newly-introduced BERT (Bidirectional Encoder Representations from Transformers) and XLNET (Extra Long Network) classifiers. We report the results achieved over one of the most popular dataset for ASAG, SciEntBank. Compared to past works for the SemEval-2013 2-way, 3-way and 5-way tasks, we obtained better or competitive performance with BERT Base (cased and uncased) and XLNET Base (cased) using a reference-based approach (considering students and model answers) and without any type of hand-crafted features.