Objective Structured Clinical Examinations (OSCEs) are typically scored by a single examiner per station, exposing results to hard-to-audit examiner variability. We developed the Rater-Aware Verification Network (RAVEN), a multimodal Artificial Intelligence (AI) system fusing egocentric video, examiner verbalisations and marks, and virtual reality (VR) action logs as an examiner-conditioned second marker for paediatric OSCEs (retrospective evaluation; 120 students, 442 ratings, eight domains). A confidence-gated hybrid improved agreement with leave-one-examiner-out consensus on the pass/fail decision (AC1: +6.2 percentage points; p = 0.0004) and domain scores (mean AC2: +3.0 points, five of eight significant), with the largest gains on borderline-fail cases (+16.3 points of concordance with consensus).
Comparing AI-inferred, rubric-specified, and examiner-articulated criteria revealed implicit practices absent from marking guidelines. Agreement is measured against a panel-derived reference rather than an external ground truth; the system is intended as an audit and flagging aid for human adjudication rather than an autonomous decision-maker. We thank the examiners and medical students who participated in the VR OSCE marking studies and the Oxford Medical Simulation team for platform support.
H.R. and A.N. acknowledge the EPSRC Turing AI Fellowship ”Ultrasound Multi-Modal Video-based Human–Machine Collaboration” [EP/X040186/1]. A.F.W. acknowledges funding from the University of Oxford Department of Engineering Science and the EPSRC Turing AI Fellowship ”Ultrasound Multi-Modal Video-based Human–Machine Collaboration” [EP/X040186/1]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Department of Engineering Science, University of Oxford, Oxford, UK Harry Rogers, Angela Feixue Wang & Alison Noble Oxford Simulation, Teaching and Research (OxSTaR), John Radcliffe Hospital, University of Oxford, Oxford, UK Sally Shiels, Ashley Tomlinson, James Thomas, Nathan Gauge, James Aylward & Helen Higham Medical Sciences Division, University of Oxford, Oxford, UK The authors declare no competing interests. Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Rogers, H., Shiels, S., Tomlinson, A. et al. An examiner-conditioned AI second marker for VR OSCEs. npj Digit. Med. (2026). https://doi.org/10.1038/s41746-026-03336-4 DOI: https://doi.org/10.1038/s41746-026-03336-4
Extract — continue reading at the source.
