Estimation of 3D anatomically précised hand poses using single shot corrective CNN

Article type: Research Article

Authors: Malavath, Pallavi^{; *} | Devarakonda, Nagaraju

Affiliations: School of Computer Science & Engineering, VIT-AP University, India

Correspondence: [*] Corresponding author. Pallavi Malavath, School of Computer Science & Engineering, VIT-AP University, India. E-mail: pallavimalavath.20phd7126@vitap.ac.in.

Abstract: A crucial component of human-computer interaction is 3D hand posture assessment. The most recent advancements in computer vision have made estimating 3D hand positions simpler by using deep sensors. The main challenge still stems from unrealistic 3D hand poses because the existing models only use the training dataset to learn the kinematic rules, which is ambiguous, and it is a difficult task to estimate realistic 3D hand poses from datasets because they are not free from anatomical errors. The suggested model in this study is trained using a closed-form expression that encodes the biomechanical rules, thus it does not entirely reliant on the pictures from the annotated dataset. This work also used a Single Shot Detection and Correction convolutional neural network (SSDC-CNN) to handle the issues in imposing anatomically correctness from the architecture level. The ResNetPlus is implemented to improve representation capability with enhanced the efficiency of error back-propagation of the network. The datasets of the Yoga Mudras, like HANDS2017, and MSRA have been used to train and test the future model. As observed from the ground truth the previous hand models have many anatomical errors but, the proposed hand model is anatomically error free hand model compared to previous hand models. By considering the ground truth hand pose, the recommended hand model has shown good accuracy when compared to the state-of-art hand models.

Keywords: Biomechanical constraints, Anatomical correction, single-shot detection and correction CNN, 3-Dimensional hand pose estimation

DOI: 10.3233/JIFS-231779

Journal: Journal of Intelligent & Fuzzy Systems, vol. 45, no. 5, pp. 8263-8277, 2023