Indonesian automatic short answer grading system

Heinrich Reagan Salim, Chintya De, Nicholas Daniel Pratamaputra, Derwin Suhartono

Abstract


Short answer question is one of the methods used to evaluate student cognitive abilities, including memorizing, designing, and freely expressing answers based on their thoughts. Unfortunately, grading short answers is more complicated than grading multiple choices answers. For that problem, several studies have tried to build an artificial intelligence system called automatic short answer grading (ASAG). We tried to improve the accuracy of the ASAG system at scoring student answers in Indonesian by enhancing the earlier state-of-the-art models and methods. They were the bidirectional encoder representations from transformer (BERT) with fine-tuning approach and ridge regression models utilizing advanced feature extraction. We conducted this study by doing stages of literature review, data set preparation, model development, implementation, and comparison. Using two different ASAG data sets, the best result of this study was an achievement of 0.9508 in pearson’s correlation and 0.4138 in root-mean-square error (RMSE) by the BERT-based model with the fine-tuning approach. This result outperformed the results of the previous studies using the same evaluation metrics. Thus, it proved our ASAG system using the BERT model with fine-tuning approach can improve the accuracy of grading short answers.

Keywords


Artificial intelligence; Automatic short answer grading; BERT; Natural language processing; Ridge regression

Full Text:

PDF


DOI: https://doi.org/10.11591/eei.v11i3.3531

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Bulletin of EEI Stats