RAG-enhanced large language model for ideological and political education: an intelligent Q&A system and its impact on college students’ employment cognition

Abstract

The digital transformation of higher education in China calls for intelligent tools that can deliver ideological and political education (IPE) content in personalized, interactive formats, and this study responds to that gap. We present an LLM-based question-answering system built on three tightly coupled design choices: a LoRA-fine-tuned ChatGLM3-6B generator grounded in a curated IPE and employment knowledge base, an intent classifier that routes queries to a dedicated employment-cognition sub-index, and an ideologically aligned three-block prompt template that keeps generation on-topic and citation-backed. Retrieval fuses BM25 with dense vectors and, in the extended baselines reported below, is optionally coupled with a cross-encoder reranker to blunt hallucination on policy-intensive queries. A quasi-experimental study involving 278 undergraduates over eight weeks evaluated the system’s impact on four dimensions of employment cognition: vocational value, employment policy, professional competence, and labor-market situation. Results show that the RAG-enhanced system achieved 89.4% answer accuracy, significantly outperforming both the unaugmented base model and a keyword-matching FAQ baseline. Students in the experimental group demonstrated statistically significant improvements across all four dimensions, with Cohen’s d ranging from 0.58 to 1.22, while the control group showed negligible change. Effect sizes were largest for knowledge-intensive dimensions and smallest for identity-level constructs. We read this pattern cautiously: an eight-week text-based intervention plausibly reshapes informational facets of employment cognition, whereas the modest shift on vocational value cognition should be interpreted as short-term informational priming rather than a genuine change in vocational identity, which is shaped by longer educational and social trajectories. The paper offers a replicable technical architecture, an evaluation protocol grounded in a four-dimensional cognition framework, and openly released prompts, LoRA configurations, retrieval scripts, and questionnaire items to support independent replication.

The authors' abstract, as published at the source. Scientific Reports, 2026 · DOI ↗