Affiliations: [a]
Department of Public Health and Preventive Medicine, School of Medicine, Jinan University, Guangzhou, China
| [b]
Division of Medical Psychology and Behaviour Science, School of Medicine, Jinan University, Guangzhou, China
| [c]
College of Information Science and Technology, Jinan University, Guangzhou, China
| [d] Rehabilitation Medicine, Second Affiliated Hospital of Nanchang University, Nanchang, China
| [e]
Neuroscience and Neurorehabilitation Institute, University of Health and Rehabilitation Science, Qingdao, China
Correspondence:
[*]
Correspondence to: Qian Tao, Division of Medical Psychology and Behavior Science, School of Medicine, Jinan University, Guangzhou, Guangdong Province, 510632, China. Tel.: +86 020 85220267; E-mail: taoqian16@jnu.edu.cn.
Note: [1] These authors contributed equally to this work.
Abstract: Background:Emotion and cognition are intercorrelated. Impaired emotion is common in populations with Alzheimer’s disease (AD) and mild cognitive impairment (MCI), showing promises as an early detection approach. Objective:We aim to develop a novel automatic classification tool based on emotion features and machine learning. Methods:Older adults aged 60 years or over were recruited among residents in the long-term care facilities and the community. Participants included healthy control participants with normal cognition (HC, n = 26), patients with MCI (n = 23), and patients with probable AD (n = 30). Participants watched emotional film clips while multi-dimensional emotion data were collected, including mental features of Self-Assessment Manikin (SAM), physiological features of electrodermal activity (EDA), and facial expressions. Emotional features of EDA and facial expression were abstracted by using continuous decomposition analysis and EomNet, respectively. Bidirectional long short-term memory (Bi-LSTM) was used to train classification model. Hybrid fusion was used, including early feature fusion and late decision fusion. Data from 79 participants were utilized into deep machine learning analysis and hybrid fusion method. Results:By combining multiple emotion features, the model’s performance of AUC value was highest in classification between HC and probable AD (AUC = 0.92), intermediate between MCI and probable AD (AUC = 0.88), and lowest between HC and MCI (AUC = 0.82). Conclusions:Our method demonstrated an excellent predictive power to differentiate HC/MCI/AD by fusion of multiple emotion features. The proposed model provides a cost-effective and automated method that can assist in detecting probable AD and MCI from normal aging.
