CNN-based Multimodal Touchless Biometric Recognition System using Gait and Speech

Issue title: Digital transformation through advances in artificial intelligence and machine learning

Guest editors: Hasmat Malik, Gopal Chaudhary and Smriti Srivastava

Article type: Research Article

Authors: Sarin, Sumit^{; *} | Mittal, Antriksh | Chugh, Anirudh | Srivastava, Smriti

Affiliations: Netaji Subhas Institute of Technology, Sector-3, Dwarka, New Delhi, India

Correspondence: [*] Corresponding author. Sumit Sarin, Netaji Subhas Institute of Technology, Sector-3, Dwarka, New Delhi, India. E-mail: sumitsarinofficial@gmail.com.

Abstract: Person identification using biometric features is an effective method for recognizing and authenticating the identity of a person. Multimodal biometric systems combine different biometric modalities in order to make better predictions as well as for achieving increased robustness. This paper proposes a touchless multimodal person identification model using deep learning techniques by combining the gait and speech modalities. Separate pipelines for both the modalities were developed using Convolutional Neural Networks. The paper also explores various fusion strategies for combining the two pipelines and shows how various metrics get affected with different fusion strategies. Results show that weighted average and product fusion rules work best for the data used in the experiments.

Keywords: Multimodal, touchless, biometric system, gait recognition, speech recognition

DOI: 10.3233/JIFS-189765

Journal: Journal of Intelligent & Fuzzy Systems, vol. 42, no. 2, pp. 981-990, 2022