Human-in-the-Loop Educational Feedback Analysis Through Topic Model Benchmarking and Generative AI Interpretation

Abstract

Open-ended Student Evaluations of Teaching (SET) provide rich information about students’ learning experiences and instructional needs, yet their large-scale analysis remains difficult to conduct through manual coding alone. This study proposes a human-in-the-loop artificial intelligence framework for transforming open-ended student feedback into structured and interpretable evidence for educational evaluation. The framework integrates manual thematic coding, natural language processing, generative AI-supported interpretation, and human expert review. Its analytical and interpretive components were evaluated using 344 comment units collected from 43 university students across four open-ended questions administered at two points during a 16-week AI-assisted blended English as a Foreign Language course. TF-IDF with k-means, Latent Dirichlet Allocation (LDA), and BERTopic were benchmarked in terms of topic coherence, topic distinctiveness, and interpretability. LDA achieved the highest coherence score (0.4224), whereas BERTopic demonstrated the highest topic distinctiveness (IRBO = 0.9889). After hyperparameter optimisation, LDA coherence increased from 0.4224 to 0.4785. The optimised topics were subsequently interpreted using generative AI and examined against manually derived themes, showing broad semantic correspondence across language skills, cognitive processes, affective factors, motivational factors, learning strategies, classroom environments, and educational technologies. The findings indicate that human-in-the-loop AI can improve the scalability and interpretability of qualitative educational feedback while retaining human oversight of contextual judgement. By combining computational analysis with generative AI-supported interpretation and expert review, the proposed framework provides a practical analytical basis for evidence-informed educational feedback and subsequent instructional decision-making.

The authors' abstract, as published at the source. Computers, 2026 · DOI ↗