Evaluating Lightweight Transformer Models for Rhetorical Element Classification in Student Essays

Abstract

As education becomes increasingly digital, automated feedback systems are an emerging technology that aim to provide timely, personalized, and high-quality feedback on students writing at a large scale. The basis of these systems is the identification of rhetorical elements in student essays, but existing approaches often use large, computationally expensive transformer models that may be impractical for classroom settings. This project aims to address the need for lightweight and effective models capable of identifying argumentative structures in student writing. To explore this, various compact transformer models (DistilBERT, TinyBERT, ELECTRA, MiniLM) are evaluated on the PERSUADE Corpus 2.0, a dataset of more than 25000 annotated argumentative essays written by students in 6th to 12th grade. Each model had to classify the rhetorical purpose of a target sentence, using preceding and following sentences as a contextual input. The models were fine-tuned using consistent hyperparameters and early stopping, and all annotations were represented using BIO tagging at the token level. THe results show that DistilBERT achieved the highest macro F1 score of 0.841, followed by MiniLM (0.822), ElectraLM (0.772), and TinyBERT (0.700). In terms of inference time, the model with the lowest inference time was TinyBERT with 2.31 ms per sentence, followed by MiniLM (40.96 ms per sentence), ElectraLM (59.06 ms per sentence), and DistilBERT (95.56 ms per sentence). These results show that MiniLM is particularly well suited for real-time, deployable feedback systems, as it balances both performance and efficiency. Future work will explore utilizing these rhetorical classification models to develop a personalized feedback system for real-time use in classrooms.