Integrating Convolutional Neural Networks and Multi-Task Dictionary Learning for Cognitive Decline Prediction with Longitudinal Images

Article type: Research Article

Authors: Dong, Qunxi^{a; 1} | Zhang, Jie^{a; 1} | Li, Qingyang^a | Wang, Junwen^b | Leporé, Natasha^c | Thompson, Paul M.^d | Caselli, Richard J.^e | Ye, Jieping^f | Wang, Yalin^{a; *} | for the Alzheimer’s Disease Neuroimaging Initiative²

Affiliations: [a] School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA | [b] Department of Health Sciences Research, Center for Individualized Medicine, Mayo Clinic, Scottsdale, AZ, USA | [c] Department of Radiology, Children’s Hospital Los Angeles, Los Angeles, CA, USA | [d] Imaging Genetics Center, Institute for Neuroimaging and Informatics, University of Southern California, Los Angeles, CA, USA | [e] Department of Neurology, Mayo Clinic Arizona, Scottsdale, AZ, USA | [f] Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA

Correspondence: [*] Correspondence to: Yalin Wang, PhD, School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, P.O. Box 878809, Tempe, AZ 85287, USA. Tel.: +1 480 965 6871; Fax: +1 480 965 2751; E-mail: ylwang@asu.edu.

Note: [1] These authors contributed equally to this work.

Note: [2] Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (http://adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.

Abstract: Background:Disease progression prediction based on neuroimaging biomarkers is vital in Alzheimer’s disease (AD) research. Convolutional neural networks (CNN) have been proved to be powerful for various computer vision research by refining reliable and high-level feature maps from image patches. Objective:A key challenge in applying CNN to neuroimaging research is the limited labeled samples with high dimensional features. Another challenge is how to improve the prediction accuracy by joint analysis of multiple data sources (i.e., multiple time points or multiple biomarkers). To address these two challenges, we propose a novel multi-task learning framework based on CNN. Methods:First, we pre-trained CNN on the ImageNet dataset and transferred the knowledge from the pre-trained model to neuroimaging representation. We used this deep model as feature extractor to generate high-level feature maps of different tasks. Then a novel unsupervised learning method, termed Multi-task Stochastic Coordinate Coding (MSCC), was proposed for learning sparse features of multi-task feature maps by using shared and individual dictionaries. Finally, Lasso regression was performed on these multi-task sparse features to predict AD progression measured by the Mini-Mental State Examination (MMSE) and the Alzheimer’s Disease Assessment Scale cognitive subscale (ADAS-Cog). Results:We applied this novel CNN-MSCC system on the Alzheimer’s Disease Neuroimaging Initiative dataset to predict future MMSE/ADAS-Cog scales. We found our method achieved superior performances compared with seven other methods. Conclusion:Our work may add new insights into data augmentation and multi-task deep model research and facilitate the adoption of deep models in neuroimaging research.

Keywords: Alzheimer’s disease, convolutional neural networks, dictionary learning, multi-task learning, transfer learning

DOI: 10.3233/JAD-190973

Journal: Journal of Alzheimer's Disease, vol. 75, no. 3, pp. 971-992, 2020