RAG-enhanced large language model for ideological and political education: an intelligent Q&A system and its impact on college students’ employment cognition

The digital transformation of higher education in China calls for intelligent tools that can deliver ideological and political education (IPE) content in personalized, interactive formats, and this study responds to that gap. We present an LLM-based question-answering system built on three tightly coupled design choices: a LoRA-fine-tuned ChatGLM3-6B generator grounded in a curated IPE and employment knowledge base, an intent classifier that routes queries to a dedicated employment-cognition sub-index, and an ideologically aligned three-block prompt template that keeps generation on-topic and citation-backed. Retrieval fuses BM25 with dense vectors and, in the extended baselines reported below, is optionally coupled with a cross-encoder reranker to blunt hallucination on policy-intensive queries.

A quasi-experimental study involving 278 undergraduates over eight weeks evaluated the system’s impact on four dimensions of employment cognition: vocational value, employment policy, professional competence, and labor-market situation. Results show that the RAG-enhanced system achieved 89.4% answer accuracy, significantly outperforming both the unaugmented base model and a keyword-matching FAQ baseline. Students in the experimental group demonstrated statistically significant improvements across all four dimensions, with Cohen’s d ranging from 0.58 to 1.22, while the control group showed negligible change.

Effect sizes were largest for knowledge-intensive dimensions and smallest for identity-level constructs. We read this pattern cautiously: an eight-week text-based intervention plausibly reshapes informational facets of employment cognition, whereas the modest shift on vocational value cognition should be interpreted as short-term informational priming rather than a genuine change in vocational identity, which is shaped by longer educational and social trajectories. The paper offers a replicable technical architecture, an evaluation protocol grounded in a four-dimensional cognition framework, and openly released prompts, LoRA configurations, retrieval scripts, and questionnaire items to support independent replication.

Recall-oriented understudy for gisting evaluation–longest common subsequence No funding was received for this research. College of Mechanical and Power Engineering, Jingjiang College, Jiangsu University, Zhenjiang, 212013, Jiangsu, China The authors declare no competing interests. This study was approved by the Institutional Review Board of Jiangsu University (Reference Number: EDU-IRB-2024-037).

All participants provided written informed consent prior to enrollment; the consent form disclosed the study purpose, the categories of data collected, the six-month raw-log retention window followed by irreversible aggregation, and the right to withdraw at any point without any impact on grades, standing in the compulsory IPE courses, or eligibility for other academic support. Access to identifiable data was limited to the research team through role-based accounts with audited reads, and the technical safeguards summarized in Sect. 3.1 were reviewed and approved by the IRB before deployment. The study was conducted in accordance with the Declaration of Helsinki and relevant national regulations.

The author has reviewed the manuscript and consents to its publication. No identifiable information regarding participants has been included. Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.