Gesture detection from RGB hand image using modified convolutional neural network

Abstract

In this paper we study the RGB-only hand gesture recognition in human computer interaction. First, the hand region and landmarks are detected, in order to remove the complex background and convert the RGB image to iconic image. Active Shape Model is adopted for landmark detection and Markov Random Fields are used for error correction. Second, we propose to use a modified CNN model to recognize the visual hand gesture information. Two stages are tested in experiments and compared with conventional machine learning models. Experimental results show that our proposed method has an advantage in recognition accuracy.