Reliability-based arbitration reveals heterogeneous model-based and model-free reinforcement learning strategies linked to compulsivity and impulsivity

The mechanisms by which humans arbitrate between reinforcement-learning strategies remain poorly understood, and the nature of these strategies has come under recent debate. Using a modified two-step decision task in two independent samples (N = 452 and N = 226), here we show that humans integrate three forms of reliability information when arbitrating control: transition reliability, model-free reward-prediction reliability, and model-based reward-prediction reliability. These mechanisms are deployed differently across individuals, revealing predominantly model-free, model-based, mixed, and non-reinforcement learning behavioral profiles that are associated with distinct compulsivity and impulsivity profiles.

Critically, individuals also differ in what they arbitrate over: some switch between model-based and model-free control, whereas others maintain model-based control while arbitrating between alternative task representations. Together, these findings reveal broad heterogeneity in human reinforcement learning and provide deeper insight into individual differences in cognition and neuropsychiatric vulnerability. This work was supported by grants from the National Institutes of Mental Health to J.P.O.D and J.C (R21MH120805 and R01MH121089) and to C.J.C (R00MH123669).

Psychological & Brain Sciences, University of Iowa, Iowa City, IA, USA Division of Humanities & Social Sciences, California Institute of Technology, Pasadena, CA, USA Jeffrey Cockburn, Weilun Ding, Reza Tadayonnejad & John P. O’Doherty Department of Psychology, University of California, Berkeley, Berkeley, CA, USA Department of Psychology, University of Maryland, College Park, MD, USA Brain and Behavior Institute, University of Maryland, College Park, MD, USA Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA Semel Institute for Neuroscience and Human Behavior at UCLA, Los Angeles, CA, USA Centre for Addiction and Mental Health, Toronto, ON, Canada Department of Psychiatry, University of Toronto, Toronto, ON, Canada Department of Women’s and Children’s Health, Karolinska Institutet, Stockholm, Sweden The authors declare no competing interests. Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material.

If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. Cockburn, J., Ding, W., Oh, S. et al.

Reliability-based arbitration reveals heterogeneous model-based and model-free reinforcement learning strategies linked to compulsivity and impulsivity. Nat Commun (2026). https://doi.org/10.1038/s41467-026-78185-6 DOI: https://doi.org/10.1038/s41467-026-78185-6