Multimodal patch-based deep learning for early Alzheimer’s disease classification using MRI and PET

Alzheimer’s disease (AD) is a neurodegenerative condition in which detection of pathological changes in the early stages, especially during the mild cognitive impairment (MCI) phase, is key for early diagnosis and treatment of the disease. While MRI gives a structural view of the brain, FDG-PET is complementary in providing a view of functional and metabolic changes in the brain due to neurodegeneration. In this study, we propose a multimodal patch-based deep learning framework for classification of AD, MCI, and cognitively normal (NC) patients based on MRI and FDG-PET data pairs.

Data consisted of 168 patients from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, which include 58 AD, 54 MCI, and 56 NC patients. T1-weighted MRI and FDG-PET volumes were preprocessed and registered to a common spatial space, followed by extraction of non-overlapping 3-D patches. We then extracted structural and functional features using two 3D convolutional neural networks (3D-CNNs), and finally multimodal fusion of features was performed.

In order to increase the robustness of the final prediction results, a stacking-based ensemble algorithm employing a 1D-CNN meta-learner was utilized for fusing the base model prediction outputs. The performance of the models was evaluated by subject-level five-fold cross-validation, while accuracy, precision, recall, F1-score, and AUC were selected as evaluation criteria. The proposed full framework reached the average accuracies of 97.50% ± 2.40%, 99.00% ± 1.90%, and 96.00% ± 2.10% for NC vs.

MCI diagnosis, respectively, achieving corresponding AUCs of 0.983, 0.996, and 0.973. Ablation analysis also proved that the patch-based representation, multimodal feature fusion, and stacking helped achieve the desired classification performance, since the full framework significantly outperformed all the ablated configurations (p < 0.01). Based on the obtained results, it is possible to claim that combining the localized structural and metabolic features provided by the MRI and FDG-PET scans within the framework of the patch-based 3D deep learning algorithm can help effectively discriminate between AD, MCI, and cognitively normal subjects.

Additional validation is necessary before applying the framework in clinical practice. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. All costs related to experiments, materials, and analyses were borne by the authors personally.

Faculty of Electrical and Computer Engineering, Babol Noshirvani University of Technology, Babol, Iran Maryam Ramzanian Keshteli & Hussain Montazery Kordy Correspondence to Hussain Montazery Kordy. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. During the preparation of this work the author(s) did not use any generative AI or AI-assisted technologies.