Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension

Machine reading comprehension tasks require a machine reader to answer questions relevant to the given document. In this paper, we present the first free-form multiple-Choice Chinese machine reading Comprehension dataset (C-3), containing 13,369 documents (dialogues or more formally written mixed-genre texts) and their associated 19,577 multiple-choice free-form questions collected from Chineseas-a-second-language examinations. We present a comprehensive analysis of the prior knowledge (i.e., linguistic, domainspecific, and generalworld knowledge) needed for these real-world problems. We implement rule-based and popular neuralmethods and find that there is still a significant performance gap between the best performing model (68.5%) and human readers (96.0%), especiallyon problems that require prior knowledge. We further study the effects of distractor plausibility and data augmentation based on translated relevant datasets for English on model performance. We expect C-3 to present great challenges to existing systems as answering 86.8% of questions requires both knowledge within and beyond the accompanying document, and we hope that C-3 can serve as a platform to study how to leverage various kinds of prior knowledge to better understand a given written or orally oriented text. C-3 is available at https://dataset.org/c3/.