Affiliations:
Department of Computer Engineering, Applied Artificial Intelligence Research Center, Near East University, North Cyprus, Turkey
Correspondence:
[*]
Corresponding author. Idoko John Bush, Department of Computer Engineering, Applied Artificial Intelligence Research Center, Near East University, North Cyprus, Via Mersin-10, Turkey. E-mail: john.bush@neu.edu.tr.
Abstract: In this study, we propose a vision-based mouse controller capable of controlling objects from a distant location via hand gestures. The proposed hybrid model constitutes hand detection, prediction of hand states and direction and finally, with the aid of deep learning algorithm, we systematically control hand gestures to reposition objects on computer screen. This hybrid system is explicitly designed to control mouse on computer screen during formal presentation. Random movement of hand from up to down and right to left move the mouse pointer and sends signal to the system utilizing states of the hand. Here, close hand places the mouse button on active mode while open hand releases the button. The proposed hybrid model is made up of two modules: Single Shot Multi Box Detection (SSD) structure utilized to detect hand while Convolutional Neural Network (CNN) is utilized for prediction. For comparative purposes, we performed similar experiment where SSD is used for hand detection while Radial Basis Function Network (RBFN) is used for hand states prediction. In the comparative results of hand states prediction, SSD+CNN greatly outperformed SSD+RBFN. The proposed hybrid model is vision-based hence, it does not require additional hardware to perform its task. Overall performance of the framework depicts that the system is accurate and robust.
Keywords: Hand gesture, convolutional neural network, radial basis function network, computer vision, deep learning
