Automated Short Answer Grading Using Bi-LSTM and Manhattan Distance

Abstract

Free-response questions are commonly used by teachers, professors, and other academic professionals to evaluate student learning on assessments. Short-answer questions especially can be highly versatile and efficient because of their open-ended nature, but can also be time-consuming to grade and provide feedback for this same reason. In existing research on short-answer question grading, machine learning systems have been developed to automate the grading process for a limited domain of questions. In this paper, we present a Bi-LSTM model that can grade any type of short-answer question, defined as an answer that is between 1-2 sentences long, regardless of the subject matter. The Bi-LSTM assigns each important word in the model, as well as each student’s answer, a vector. Each combination of words is given a similarity score using the Manhattan Distance between 0 and 1, and thus can be assigned a grade as a percentage. We utilized 3 types of datasets: the first one had answers of a couple of words, the second had answers of a couple of phrases, and the last dataset included student answers with multiple sentences and given grades. Using the datasets, we found a mean accuracy of approximately 87% through ten trials for dataset 1, 91% for dataset 2, and 89% for dataset 3 using the logistic regression model. The results show that our model was able to achieve similar success as compared to human graders. One limitation is that the method cannot be applied to longer answers.