Accuracy of MRI Classification Algorithms in a Tertiary Memory Center Clinical Routine Cohort

Article type: Research Article

Authors: Morin, Alexandre^{a; b; c; 1} | Samper-Gonzalez, Jorge^{b; c; 1} | Bertrand, Anne^{b; c; d; †} | Ströer, Sébastian^{b; d} | Dormont, Didier^{b; c; d} | Mendes, Aline^e | Coupé, Pierrick^f | Ahdidan, Jamila^g | Lévy, Marcel^e | Samri, Dalila^e | Hampel, Harald^{e; h; i} | Dubois, Bruno^{e; i} | Teichmann, Marc^{e; i} | Epelbaum, Stéphane^{b; c; e} | Colliot, Olivier^{b; c; d; e; *}

Affiliations: [a] Department of Neurology, AP-HP, Hôpital de la Pitié-Salpêtrière, Unité de Neuro-Psychiatrie Comportementale (UNPC), Paris, France | [b] Sorbonne Universités, UPMC Univ Paris 06, Inserm, CNRS, ICM, Paris, France | [c] Inria, Aramis-Project Team, Paris, France | [d] Department of Neuroradiology, AP-HP, Hôpital de la Pitié-Salpêtrière, Paris, France | [e] Department of Neurology, AP-HP, Hôpital de la Pitié-Salpêtrière, Institut de la Mémoire et de la Maladie d’Alzheimer (IM2A), Paris, France | [f] Laboratoire Bordelais de Recherche en Informatique, Unit Mixte de Recherche CNRS (UMR 5800), PICTURA Research Group, Bordeaux, France | [g] Brainreader, Horsens, Denmark | [h] AXA Research Fund and UPMC Chair, Paris, France; Sorbonne Universities, Pierre et Marie Curie University, Paris, France | [i] ICM, ICM-INSERM 1127, FrontLab, Paris, France

Correspondence: [*] Correspondence: Olivier Colliot, PhD, ICM – Brain and Spinal Cord Institute, ARAMIS team, Pitié-Salpêtrière Hospital, 47-83, boulevard de l’Hôpital, 75651 Paris Cedex 13, France. Tel.: +33 01 57 27 43 65; E-mail: olivier.colliot@upmc.fr.

Note: [†] Deceased: March 2, 2018.

Note: [1] Statistical analysis conducted by Dr. Alexandre Morin, MD, UPMC, AramisLab and Jorge Samper, AramisLab.

Abstract: Background:Automated volumetry software (AVS) has recently become widely available to neuroradiologists. MRI volumetry with AVS may support the diagnosis of dementias by identifying regional atrophy. Moreover, automatic classifiers using machine learning techniques have recently emerged as promising approaches to assist diagnosis. However, the performance of both AVS and automatic classifiers have been evaluated mostly in the artificial setting of research datasets. Objective:Our aim was to evaluate the performance of two AVS and an automatic classifier in the clinical routine condition of a memory clinic. Methods:We studied 239 patients with cognitive troubles from a single memory center cohort. Using clinical routine T1-weighted MRI, we evaluated the classification performance of: 1) univariate volumetry using two AVS (volBrain and Neuroreader™); 2) Support Vector Machine (SVM) automatic classifier, using either the AVS volumes (SVM-AVS), or whole gray matter (SVM-WGM); 3) reading by two neuroradiologists. The performance measure was the balanced diagnostic accuracy. The reference standard was consensus diagnosis by three neurologists using clinical, biological (cerebrospinal fluid) and imaging data and following international criteria. Results:Univariate AVS volumetry provided only moderate accuracies (46% to 71% with hippocampal volume). The accuracy improved when using SVM-AVS classifier (52% to 85%), becoming close to that of SVM-WGM (52 to 90%). Visual classification by neuroradiologists ranged between SVM-AVS and SVM-WGM. Conclusion:In the routine practice of a memory clinic, the use of volumetric measures provided by AVS yields only moderate accuracy. Automatic classifiers can improve accuracy and could be a useful tool to assist diagnosis.

Keywords: All cognitive disorders/dementia, Alzheimer’s disease, assessment of cognitive disorders/dementia, magnetic resonance imaging, volumetric MRI

DOI: 10.3233/JAD-190594

Journal: Journal of Alzheimer's Disease, vol. 74, no. 4, pp. 1157-1166, 2020