Zhixin: dataset construction and instruction fine-tuning for LLM-based mental disorder diagnosis

Automatic diagnosis of mental disorder from chief complaints is an appealing but challenging task. Large Language Models (LLMs) have shown rich knowledge and strong reasoning ability in medical domain. However, mental disorder diagnosis based on chief complaints requires highly specialised knowledge.

Lack of corresponding resources hinders LLMs performance on this task. To address this problem, we collect 6,780 complaint-diagnosis pairs spanning 29 different types of mental disorders from a specialized hospital as the dataset. Meanwhile we conduct detailed data cleansing and manual proofreading to protect the privacy.

On this basis, we propose the model named Zhixin, a LLM for mental disorder diagnosis fine-tuned on the proposed dataset. The experimental results show that Zhixin outperforms the state-of-the-art approaches in terms of automatic evaluation. It surpasses 3-shot DeepSeek-R1 by 18.54\(\%\) on Overall Accuracy.

Meanwhile, human evaluation results show that the generated response from Zhixin ensures the Accuracy, Safety and Professionalism, which are critical to the diagnosis of mental disorder. In-depth analysis of category-wise performance sheds light on the future research direction, suggesting the focus on relatively rare metal disorders. This work was supported by Research on Key Technologies for Building a Multi modal Large Model Mental Health Profile by Integrating EEG Signals(LJ212413631015) , Technology Innovation Project of Dalian Neusoft University of Information [TIFP202306] and Key Scientific Research Project of the Anhui Provincial Department of Education, China, under Grant No. 2025AHGXZK30479.

School of Health Care Technology, Dalian Neusoft University of Information, Dalian, 116033, China Xiujie Zhao, Fengdong Sun, Yanrong Dong & Yubo Wang School of Big Data and Artificial Intelligence, Fuyang University of Technology, Fuyang, 236000, China The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. For in-context learning, we implement 0-shot (no example) scenario and 3-shot (3 examples) scenarios, with corresponding prompts presented in Tables 5 and 6.

As reference, we also give the English translation. DSM-5 strives to fulfill the need of clinicians, patients, families, and researchers for a clear and concise description of each mental disorder, which has been operationalized using diagnostic criteria that are supplemented by dimensional measures of severity and is accompanied by a digest of information about the diagnosis, including risk factors and culture-and sex- and gender-related issues. For each mental disorder, DSM-5 introduces Diagnostic Features, Development and Course, Risk and Prognostic Factors, Differential Diagnosis, Comorbidity, etc5.