Large language models are increasingly used in settings where confident but incorrect answers can mislead users. Reliable uncertainty communication requires a form of metacognition: monitoring when one’s own answers are likely to be correct. Yet models’ stated confidence is often poorly aligned with answer correctness.
We test whether supervised fine-tuning improves uncertainty communication and whether gains transfer across domains and task formats. We fine-tuned two models on general knowledge, mathematics, and open-ended trivia questions. We evaluated single-question confidence estimation, in which the model reports numeric confidence for one answer, and pairwise confidence comparison, in which it chooses which of two questions it is more likely to answer correctly.
We tested held-out questions from training domains and new medical, legal, and truthfulness benchmarks. We assessed calibration, discrimination, and answer accuracy before and after fine-tuning. Here we show that fine-tuning improves alignment between stated confidence and observed accuracy and increases the model’s ability to assign higher confidence to correct than to incorrect answers.
Gains occur within training domains and, to a lesser extent, in new domains. However, single-task training does not reliably transfer between single-question confidence estimation and pairwise confidence comparison. Multitask fine-tuning produces broader gains in the models and tasks studied here.
Uncertainty communication in large language models is trainable, but transfer across metacognitive tasks is limited. Joint training on multiple confidence tasks may support broader generalization, although further tests across model families and metacognitive tasks are needed. What would it take for a language model not only to generate an answer but also to assess how likely that answer is to be correct?
As Large Language Models (LLMs) are increasingly deployed in education, business, law, and medicine, their responses shape decisions with real-world consequences. However, these models sometimes produce factually incorrect information, and users may act on such errors when they are presented without clear signals of uncertainty1. Such failures have already been documented in high-stakes legal contexts2,3 as well as in medical applications4,5.
Extract — continue reading at the source.
