Abstract

A single-modality approach to neurodegenerative disease classification is often limited by heterogeneous clinical manifestations, modality-specific feature distributions, and incomplete data availability. This study proposes an Adaptive Multimodal Meta-Fusion (AMMF) framework integrating cognitive-demographic, structural MRI, and EEG biomarkers for classification of the Alzheimer's disease (AD), Frontotemporal Dementia (FTD), and Cognitively Normal (CN) participants. Three modality-specific cohorts from two public repositories were considered: cognitive-demographic data (n = 679), structural MRI data (n = 595), and EEG data (n = 194; BrainLat n = 106 and OpenNeuro n = 88). Cognitive-demographic features are prioritized using explainable XGBoost with SHAP-based feature analysis, MRI data are processed using ConvNeXt-Small for spatial representation learning, and EEG signals are analyzed using a Transformer-based architecture for temporal and functional representation learning. For the matched BrainLat cohort, the three modality branches generated an 18-dimensional cognitive-demographic representation, a 128-dimensional EEG representation, and a 768-dimensional MRI representation. The proposed framework then applied adaptive late fusion at the modality-probability level: each modality generated an independent three-class probability vector, and a modality-availability mask was used to renormalize the predefined modality weights of 0.20, 0.30, and 0.50 assigned to the cognitive-demographic, EEG, and MRI modalities, respectively. The resulting weighted class-probability vector was supplied to a Random Forest meta-classifier for final three-class classification. The complete multimodal evaluation was performed on 106 subject-level matched participants with all three modalities. The proposed framework achieved an accuracy of 99.06%, demonstrating the utility of combining complementary cognitive, functional, and structural biomarkers through adaptive probability-level meta-fusion for AD, FTD, and CN classification. A continuity-corrected McNemar test yielded χ² = 4.000 (p = 0.0455) for C4 versus C7 and χ² = 4.167 (p = 0.0412) for C5 versus C7, indicating statistically significant paired differences, while C6 versus C7 yielded χ² = 3.200 (p = 0.0736). Integration of cognitive-demographic, MRI, and EEG modalities for multimodal disease classification. The proposed framework integrates cognitive-demographic, MRI, and EEG modalities for multimodal disease classification by combining heterogeneous features in a unified representation space and applying Random Forest-based meta-classification to the fused modality probabilities.

Keywords

Multimodal Learning, Neurodegenerative Disease Classification, EEG Signal Analysis, MRI Deep Learning, Xgboost With SHAP, Convnext-Small Network, Transformer Model, Feature Fusion, Random Forest Classifier,

Downloads

Download data is not yet available.

References

  1. X. Liu, P.S. Jones, M. Pasternak, M. Masellis, A. Bouzigues, L.L. Russell, P.H. Foster, E. Ferry-Bolder, J. van Swieten, L. Jiskoot, H. Seelaar, Frontoparietal network integrity supports cognitive function in pre‐symptomatic frontotemporal dementia: Multimodal analysis of brain function, structure, and perfusion. Alzheimer's & Dementia, 20(12), (2024) 8576-8594. https://doi.org/10.1002/alz.14299
  2. G. Marti-Juan, M. Lorenzi, G. Piella, MC-RVAE: Multi-channel recurrent variational autoencoder for multimodal Alzheimer’s disease progression modelling. NeuroImage, 268, (2023) 119892. https://doi.org/10.1016/j.neuroimage.2023.119892
  3. J. Li, W. Liang, X. Yin, J. Li, W. Guan, Multimodal gait abnormality recognition using a convolutional neural network–bidirectional long short-term memory (CNN-BiLSTM) network based on multi-sensor data fusion. Sensors, 23(22), (2023) 9101. https://doi.org/10.3390/s23229101
  4. M. Neumann, H. Kothare, V. Ramanarayanan, Multimodal speech biomarkers for remote monitoring of ALS disease progression. Computers in Biology and Medicine, 180, (2024) 108949. https://doi.org/10.1016/j.compbiomed.2024.108949
  5. J. Zhang, H. Wang, Y. Zhao, L. Guo, L. Du, Identification of multimodal brain imaging association via a parameter decomposition based sparse multi-view canonical correlation analysis method. BMC Bioinformatics, 23(Suppl 3), (2022) 128. https://doi.org/10.1186/s12859-022-04669-z
  6. X. Chen, H. Xie, X. Tao, F.L. Wang, M. Leng, B. Lei, Artificial intelligence and multimodal data fusion for smart healthcare: topic modeling and bibliometrics. Artificial Intelligence Review, 57(4), (2024) 91. https://doi.org/10.1007/s10462-024-10712-7
  7. P. Lu, L. Hu, A. Mitelpunkt, S. Bhatnagar, L. Lu, H. Liang, A hierarchical attention-based multimodal fusion framework for predicting the progression of Alzheimer’s disease. Biomedical Signal Processing and Control, 88, (2024) 105669. https://doi.org/10.1016/j.bspc.2023.105669
  8. D. Machado Reyes, H. Chao, J. Hahn, L. Shen, P. Yan, Identifying progression-specific Alzheimer’s subtypes using multimodal transformer. Journal of Personalized Medicine, 14(4), (2024) 421. https://doi.org/10.3390/jpm14040421
  9. X. Chen, S. Wang, W. Kong, Multimodal data fusion for Alzheimer's disease based on dynamic heterogeneous graph convolutional neural network and generative adversarial network. Array, 26, (2025) 100415. https://doi.org/10.1016/j.array.2025.100415
  10. X.H. Shi, L. Ju, L. Dong, R.H. Zhang, L. Shao, Y.N. Yan, Y.X. Wang, X.F. Fu, Y.Z. Chen, Z.Y. Ge, W.B. Wei, Deep learning models for the screening of cognitive impairment using multimodal fundus images. Ophthalmology Retina, 8(7), (2024) 666-677. https://doi.org/10.1016/j.oret.2024.01.019
  11. H. Guo, X. Shen, M. Lyu, G. Zhou, C. Chen, B. Xing, Y. Xie, Research progress of multimodal biomarkers in the early diagnosis of mild cognitive impairment in Parkinson’s disease. Frontiers in Neurology, 16, (2025) 1652378. https://doi.org/10.3389/fneur.2025.1652378
  12. F. Albrecht, K. Poulakis, M. Freidle, H. Johansson, U. Ekman, G. Volpe, E. Westman, J.B. Pereira, E. Franzén, Unraveling Parkinson's disease heterogeneity using subtypes based on multimodal data. Parkinsonism & Related Disorders, 102, (2022) 19-29. https://doi.org/10.1016/j.parkreldis.2022.07.014
  13. J. Wang, S. Wen, W. Liu, X. Meng, Z. Jiao, Deep joint learning diagnosis of Alzheimer’s disease based on multimodal feature fusion. BioData Mining, 17(1), (2024) 48. https://doi.org/10.1186/s13040-024-00395-9
  14. S. Benredjem, T. Mekhaznia, R. Abdulghafor, S. Turaev, A. Bennour, B. Sofiane, A. Aborujilah, M. Al Sarem, Parkinson’s disease prediction: An attention-based multimodal fusion framework using handwriting and clinical data. Diagnostics, 15(1), (2025) 4. https://doi.org/10.3390/diagnostics15010004
  15. U.M. Akkaya, H. Kalkan, A new approach for multimodal usage of gene expression and its image representation for the detection of Alzheimer’s disease. Biomolecules, 13(11), (2023) 1563. https://doi.org/10.3390/biom13111563
  16. S. Qiu, M.I. Miller, P.S. Joshi, J.C. Lee, C. Xue, Y. Ni, Y. Wang, I.D. Anda-Duran, P.H Hwang, J.A. Cramer, B.C. Dwyer, H.Hao, M.C. Kaku, S. Kedar, P.H. Lee, A.Z. Mian, D.L. Murman, S.O'Shea, A.B. Paul, M.H. Saint-Hilaire, E.A. Sartor, A.R. Saxena, L.C. Shih, J.E. Small, M.J. Smith, A. Swaminathan, C.E. Takahashi, O. Taraschenko, H. You, J. Yuan, Y. Zhou, S. Zhu, M.L Alosco, J. Mez, T.D. Stein, K.L Poston, R. Au, V. B Kolachalama, Multimodal deep learning for Alzheimer’s disease dementia assessment, Nature Communications 13(1) (2022) 3404. https://doi.org/10.1038/s41467-022-31037-5.
  17. A.A. Soladoye, N. Aderinto, D. Osho, D.B. Olawade, Explainable machine learning models for early Alzheimer’s disease detection using multimodal clinical data. International Journal of Medical Informatics, (2025) 106093. https://doi.org/10.1016/j.ijmedinf.2025.106093
  18. Q.A. Duong, S.K. Kim, J.K. Gahm, Multimodal 3D deep learning for early diagnosis of Alzheimer’s disease. IEEE Access, 12, (2024) 43120-43134. https://doi.org/10.1109/ACCESS.2024.3381862
  19. A. Francesconi, L. di Biase, D. Cappetta, F. Rebecchi, P. Soda, R. Sicilia, V. Guarrasi, Class balancing diversity multimodal ensemble for Alzheimer’s disease diagnosis and early detection. Computerized Medical Imaging and Graphics, 123, (2025) 102529. https://doi.org/10.1016/j.compmedimag.2025.102529
  20. P. Zhuang, J. Liu, X. Cai, B. Wu, Association between cognitive impairment and multimodal MRI markers in patients with cerebral microbleeds: A retrospective case–control study. International Journal of General Medicine, 19, (2026) 556275. https://doi.org/10.2147/IJGM.S556275
  21. P.D. Kuate, J.V.N. Tegnitsap, Z.T. Njitacke, M. Zhao, T.F. Fozin, P.A. Valdes-Sosa, H. Ito, M. Frasca, F. D’Agata, L. Minati, Electronic emulation of neurodegenerative disease progression on a CMOS-based physical model of human connectome. IEEE Access, 14, (2026) 39232-39260. https://doi.org/10.1109/ACCESS.2026.3670097
  22. R.V. Anand, T.S. Priyan, M.G. Brahmam, B. Balusamy, F. Benedetto, IMNMAGN: integrative multimodal approach for enhanced detection of neurodegenerative diseases using fusion of multidomain analysis with graph networks. IEEE Access, 12, (2024) 73095-73112. https://doi.org/10.1109/ACCESS.2024.3403860
  23. N. Xu, C. Wang, L. Peng, X.H. Zhou, J. Chen, Z. Cheng, Z.G. Hou, A graph-based multimodal fusion framework for assessment of freezing of gait in Parkinson’s disease. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 33, (2025) 1539-1549. https://doi.org/10.1109/TNSRE.2025.3561942
  24. Y. Bouter, R.M. Glasnek, J.M. Wenzel, C. Bouter, 18F-FDG-PET and multimodal biomarker integration: a powerful tool for Alzheimer’s disease diagnosis. Nuclear Medicine and Molecular Imaging, 59, (2025) 453-471. https://doi.org/10.1007/s13139-025-00932-2