A BERT-based modular framework for automated English essay scoring via trait analysis

Abstract

Automated essay scoring (AES) systems are commonly implemented using holistic scoring, which limits interpretability and prevents assessment at the writing trait level. As a result, such systems provide limited diagnostic and actionable feedback. To address this limitation, this study proposes a modular trait-based AES framework that separates structure and grammar evaluation while maintaining an integrated scoring mechanism. The proposed framework consists of two modules. The structure module evaluates the ideas, organization, and style traits using a bidirectional encoder representations from transformer-bidirectional long short-term memory (BERT-BiLSTM-Attention) architecture trained on the automated student assessment prize (ASAP) dataset. The grammar module evaluates the Conventions trait by applying a BERT-based grammatical acceptability classifier trained on the Corpus of linguistic acceptability (CoLA) dataset, followed by multinomial logistic regression to convert grammatical patterns into interpretable grammar scores. Experiments were conducted on the ASAP dataset and evaluated using the quadratic weighted Kappa (QWK) metric. The structure module achieved a QWK score of 0.7906 on the test set, while the grammar module obtained a QWK of 0.3923. The integrated holistic score reached a QWK of 0.7847. These results demonstrate that the proposed modular framework improves interpretability and scoring performance, supporting more objective and actionable essay evaluation for formative assessment in English language education.