Intelligent educational question–answer systems for consultation and assessment: methods, challenges, and prospects

Rakhila Turebayeva, Rozamgul Niyazova, Aktayeva Alimbubi, Denis Pleskachev, Zhanyl Sarsenbayeva, Musatai Esmaganbet

Abstract


Automated assessment of open-ended responses is challenging in multilingual and low-resource educational settings because annotated corpora and language-specific semantic resources are limited. Educational question–answer systems often separate psychometric modelling from natural language processing (NLP), restricting the integration of ability estimation and semantic interpretation. This study developed and evaluated an interpretable hybrid framework that combines three-parameter item response theory (IRT) adaptive testing with contextual semantic evaluation using language embeddings and cosine similarity. The framework integrates adaptive item selection, response interpretation, ability estimation, and diagnostic feedback. KazQAD, a Kazakh question-answer dataset comprising over 12,000 annotated pairs, was developed to support this study. An evaluation involving 350 undergraduates compared fixed-form computer-based testing (CBT), standalone IRT-based adaptive testing, and the hybrid framework. Among the three approaches, the hybrid framework showed the strongest correlation with expert assessment (r=0.91), the lowest mean standard error of ability estimation (SE=0.08), and the shortest average assessment duration (23 minutes vs 42 minutes for fixed-form CBT), a 45.2% reduction. These findings support integrating psychometrically calibrated estimation with semantic interpretation within a unified adaptive workflow for Kazakh-language assessment. However, this study did not directly benchmark contemporary transformer-based semantic graders, and the findings remain specific to the evaluated conditions and do not establish universal superiority.

Keywords


Adaptive testing; Cosine similarity metric; Digital learning environments; Intelligent educational systems; Item response theory; Knowledge assessment; Question difficulty modelling

Full Text:

PDF


DOI: https://doi.org/10.11591/eei.v15i6.11787

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Bulletin of EEI Statistics

Bulletin of Electrical Engineering and Informatics (BEEI)
ISSN: 2089-3191 , e-ISSN: 2302-9285
This journal is published by the Institute of Advanced Engineering and Science (IAES) .