Classification of mathematical test questions using machine learning on datasets of learning management system questions

Every student has a varied level of mathematical proficiency. Therefore, it is important to provide them with questions accordingly. Owing to advances in technology and artificial intelligence, the Learning Management System (LMS) has become a popular application to conduct online learning for students. The LMS can store multiple pieces of information on students through an online database, enabling it to recommend appropriate questions for each student based on an analysis of their previous responses to questions. Particularly, the LMS manages learners and provides an online platform that can evaluate their skills. Questions need to be classified according to their difficulty level so that the LMS can recommend them to learners appropriately and thereby increase their learning efficiency. In this study, we classified large-scale mathematical test items provided by ABLE Tech, which supports LMS-based online mathematical education platforms, using various machine learning techniques according to the difficulty levels of the questions. First, through t-test analysis, we identified the significant correlation variables according to the difficulty level. The t-test results showed that answer rate, type of question, and solution time were positively correlated with the difficulty of the question. Second, items were classified according to their difficulty level using various machine learning models, such as logistic regression (LR), random forest (RF), and extreme gradient boosting (xgboost). Accuracy, precision, recall, F1 score, the area under the curve of the receiver operating curve (AUC-ROC), Cohen's Kappa and Matthew's correlation coefficient (MCC) scores were used as the evaluation metrics. The correct answer rate, question type, and time for solving a question correlated significantly with the difficulty level. The machine learning-based xgboost model outperformed the statistical machine learning models, with a 85.7% accuracy, and 85.8% F1 score. These results can be used as an auxiliary tool in recommending suitable mathematical questions to various learners based on their difficulty level.