Semantic structure and measurement in large language models for psychopathology and psychometrics

Clinical assessment and empirical research in psychiatry rely heavily on standardized questionnaires to operationalize psychopathology. In a recent and timely contribution, Kambeitz et al. demonstrate that large language models (LLMs) capture substantial components of the empirical structure of psychopathology by reconstructing item–item associations and subdomain clustering solely from the semantic and sentiment embeddings of questionnaire items1. Their findings compellingly show that the low-dimensional structure traditionally attributed to latent psychopathological traits is, to a non-trivial extent, reflected in the linguistic properties of the instruments used to measure them.

We argue that this result has broader epistemological and methodological implications than are explicitly articulated, particularly for how psychometric measurement is conceptualized and how future assessment tools might be designed. We suggest that the representational correspondence identified by Kambeitz et al.1 can be extended to a semantic account of measurement itself, an interpretation supported by convergent empirical evidence from recent work on semantic psychometrics and generative language models2. Kambeitz et al. analyze multiple large-scale datasets and show consistent correlations between empirically observed item–item associations and those derived from LLM-based embeddings, with random forest models predicting empirical associations with moderate to high accuracy1.

Notably, they demonstrate partial reconstruction of empirical clustering and subscale structures across established psychopathology questionnaires. These findings reinforce the primacy of language in psychiatric assessment: symptoms are described verbally, questionnaires are composed of linguistic items, and empirical covariance structures emerge from responses to these descriptions. The study therefore provides strong evidence that LLMs encode meaningful aspects of psychopathological structure as it is operationalized in current instruments.

This is a preview of subscription content, access via your institution Receive 12 digital issues and online access to articles Prices may be subject to local taxes which are calculated during checkout Kambeitz, J. et al. Health 3, 1482–1492 https://doi.org/10.1038/s44220-025-00527-y (2025). Rep. 15, 37313 https://doi.org/10.1038/s41598-025-21289-8 (2025).

Measuring the Mind: Conceptual Issues in Contemporary Psychometrics (Cambridge Univ. Behav. 9, 944–954 https://doi.org/10.1038/s41562-024-02089-y (2025). Article PubMed PubMed Central Google Scholar Arnulf, J.

Psychol. 15, 1308098 https://doi.org/10.3389/fpsyg.2024.1308098 (2024). Health 4, e0000848 https://doi.org/10.1371/journal.pdig.0000848 (2025). In Proc. 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2025) 242–248 (Association for Computational Linguistics, 2025).