Recognition of dynamic gestures in arabic sign language using two stages hierarchical scheme

Article type: Research Article

Authors: Al-Rousan, Mohammed^{; *} | Al-Jarrah, Omar | Al-Hammouri, Mohammed

Affiliations: Department of Computer Engineering, Jordan University of Science and Technology, Irbid, Jordan

Correspondence: [*] Corresponding author. E-mail: alrousan@just.edu.jo

Abstract: Most of the existing work on Arabic sign language (ArSL) recognition focuses on static gestures, while there is a growing need for recognition of continuous gestures. In this work, we develop a system that makes automatic translation of dynamic gestures in the Arabic Sign Language (ArSL) using two stages (Hierarchical) scheme. The system is composed of two stages: the first stage recognizes the group of the gesturer and the second stage recognizes the gestures within the groups. Spatial domain analysis is used for features extraction from the hands and face regions, which are classified using Hidden Markov Model (HMM). The extracted features include eccentricity of the hand region, coordinate of the center of the hand region, direction angle of the hand region, and the hand vector that represents the shape of the hand. These features are scale and translation invariant. We have used two types of features: simple and complex. The simple features comprise six features and the complex comprises 17 features. The complex features include 11 hand vectors which are not included in the simple features. The recognition rate for the signer-dependent is 92.5% and for the signer-independent is 70.5%.

Keywords: Sign language recognition, arabic sign language, hidden markov model, spatial domain analysis, dynamic gestures

DOI: 10.3233/KES-2010-0197

Journal: International Journal of Knowledge-based and Intelligent Engineering Systems, vol. 14, no. 3, pp. 139-152, 2010