Synthetic Training of Deep CNN for 3D Hand Gesture Identification

In this paper, we present some experiments and investigations on a synthetically-trained neural network for the 3D hand gesture identification problem. The training process of a deep-learning neural network typically requires a large amount of training data to converge to a valid recognition model. However, in practice, it is difficult to obtain a large set of tagged real-data for the training purposes. In this paper, we investigate the plausibility of combining a large set of computer-generated 3D hand images with few real-camera images to form the training data set for the 3D hand gesture recognition applications. It is shown that by adding 0.09% of real images to the synthetic training data set, the recognition accuracy are raised from 37.5% to 77.08% for the problem of identifying 24 classes of hand gestures of an unknown user whose hand was not used in the training data set. In this paper, we have shown that the effect of the few real images to the trained CNN models mainly falls upon the fully-connected layers.