Simultaneous multi-task assessment of five neurological language disorders via cross-corpus multimodal learning

Automated analysis of spontaneous speech may offer a scalable, objective complement to clinical assessment of neurological language disorders, yet existing approaches are largely corpus- and disease-specific. We propose a unified model that simultaneously classifies five neurological language disorders—Aphasia, Alzheimer’s Disease (AD), Mild Cognitive Impairment (MCI), Traumatic Brain Injury (TBI), and Right Hemisphere Damage (RHD)—and a healthy Control class across five heterogeneous TalkBank corpora (1,638 samples). To our knowledge, this is the first model to combine multimodal input (text, speech, and tabular metadata), multi-corpus training, and multi-task output for this many disorders, fusing modalities through a Q-Former cross-attention module that feeds a shared representation to a classifier and three auxiliary clinical heads (WAB-AQ severity regression, disease-similarity retrieval, and longitudinal progression).

Under five-seed evaluation the model attains test accuracy \(90.98\%\) and Macro-F1 \(86.19\%\), outperforming all baselines, and the auxiliary heads estimate aphasia severity within the WAB-AQ’s measurement error, retrieve same-disorder patients, and track recovery. Ablation identifies Q-Former fusion and class-frequency-weighted sampling as the principal performance drivers, and an inter-disorder correlation analysis explains the dominant MCI–Control confusion, recovers an etiology-based disorder grouping, and finds paralinguistic features among the top discriminators. Leave-One-Corpus-Out evaluation shows that zero-shot generalization to unseen disease classes remains fundamentally challenging.

This research was supported by the National Research Council of Science & Technology (NST) grant funded by the Korean government (MSIT) (No. CAP21055-300) and by the Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean government (No. 26CB1200). Integrated Intelligence Research Section, Electronics and Telecommunications Research Institute, Daejeon, 34129, Republic of Korea Byounghwa Lee, Hwa Jeon Song & Byung Ok Kang The authors declare no competing interests.

This work used only existing, publicly available de-identified data from TalkBank. Therefore, no approval or waiver from an institutional review board (IRB) was required. Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material.

If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. Lee, B., Song, H.J. & Kang, B.O.