Recognize basic emotional statesin speech by machine learning techniques using mel-frequency cepstral coefficient features

Issue title: Special Section: Applied Machine Learning and Management of Volatility, Uncertainty, Complexity & Ambiguity (V.U.C.A)

Guest editors: Srikanta Patnaik

Article type: Research Article

Authors: Yang, Ningning^a | Dey, Nilanjan^b | Sherratt, R. Simon^c | Shi, Fuqian^{a; *}

Affiliations: [a] First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China | [b] Department of Information Technology, Techno India College of Technology, West Bengal, India | [c] Department of Biomedical Engineering, The University of Reading, UK

Correspondence: [*] Corresponding author. Fuqian Shi, First Affiliated Hospital of Wenzhou Medical University, Wenzhou 325035, China. E-mail: sfq@wmu.edu.cn.

Abstract: Speech Emotion Recognition (SER) has been widely used in many fields, such as smart home assistants commonly found in the market. Smart home assistants that could detect the user’s emotion would improve the communication between a user and the assistant enabling the assistant to offer more productive feedback. Thus, the aim of this work is to analyze emotional states in speech and propose a suitable algorithm considering performance verses complexity for deployment in smart home devices. The four emotional speech sets were selected from the Berlin Emotional Database (EMO-DB) as experimental data, 26 MFCC features were extracted from each type of emotional speech to identify the emotions of happiness, anger, sadness and neutrality. Then, speaker-independent experiments for our Speech emotion Recognition (SER) were conducted by using the Back Propagation Neural Network (BPNN), Extreme Learning Machine (ELM), Probabilistic Neural Network (PNN) and Support Vector Machine (SVM). Synthesizing the recognition accuracy and processing time, this work shows that the performance of SVM was the best among the four methods as a good candidate to be deployed for SER in smart home devices. SVM achieved an overall accuracy of 92.4% while offering low computational requirements when training and testing. We conclude that the MFCC features and the SVM classification models used in speaker-independent experiments are highly effective in the automatic prediction of emotion.

Keywords: Emotion recognition, back propagation neural network, extreme learning machine, Mel-frequency cepstral coefficients, smart home, support vector machine

DOI: 10.3233/JIFS-179963

Journal: Journal of Intelligent & Fuzzy Systems, vol. 39, no. 2, pp. 1925-1936, 2020