Real-time Visual-based Hand Gesture Recognition System And Its FPGA-based Hardware Implementation

Gesture recognition has been paid more and more attention as a new generationof human-computer interaction and visual input mode. Gesture recognition is playingan important role in human-computer intention, information exchange, interactiveentertainment, demonstration study, etc. So the research and development is becomingincreasingly important.However, because of its inherent features such as diversity, ambiguity,space-time difference and large computational burden, it is difficult to achieve thereal-time application with software, especially in embedded system.Therefore, in this paper we propose a real-time hardware-based gesturerecognition system as well as innovative algorithm architecture, which is easy toimplement by hardware. Moreover, the operating speed is improved significantly. Byreplacing the main recognition algorithm module, this architecture can be used inthree different directions, namely, static hand recognition, dynamic gesturerecognition and hand trajectory recognition. The architecture is FPGA-basedhardware implementation, and the system is implemented on a single chip (namely asystem on chip or SOC) with low cost and high performance. When we look at theentire system, first of all, the system utilizes the camera to capture the image data withYCrCb format and then calculate the probability of each pixel point coincident withstandard color pixel through the Gauss probability model. Secondly, through thethreshold limit, the24bit data YCrCb is simplified into the binary image data. Thetransformed binary image data simplifies the processing complexity of the algorithmand greatly reduce hardware resource consumption at the same time. Then the systemuses the mean filter to deal with the binary image data disposed by Gauss probabilitymodel for noise reduction. The filter not only removes white noise of CMOS imagesensor but also filters most environmental noise introduced by lighting. The binaryimage data disposed by the median filter needs to be output from the two paths. One way is stored in Ping-Pong RAM and then shown on the display and the other way isused for positioning and image segmentation. Finally, the data stored in Ping-PongRAM is used as gesture recognition algorithm input and meanwhile shown on display,so as to realize the synchronization between image display and gesture recognition.As this architecture use parallel and pipeline structure, the operating speed of gesturerecognition system based on FPGA improved significantly, which can achieves60frames per second. The good real-time performance of this gesture recognition systemis also attributed to the overall use of synchronous clock design.In this paper，we propose a novel static hand recognition algorithm. It optimizesthe existing fingertip detection algorithms with the proposed essential constraints,which improves the recognition rate to90.3%. Using parallel and pipeline structure,the gesture recognition system based on FPGA achieves60frames per second.Dynamic gesture recognition is considered as a sound solution to the HumanComPuter Inieraetion(HCI). The architecture which is based on FPGA complete thereal-time dynamic hand gesture recognition with emphasis on the method of hardwareimplementation. The60fps dynamic gestures in the video stream can be identified inthis system using parallel structure. In trajectory recognition module, it adopts thedescription method of state machine, which can decide the direction of gesturemovement within the scope of the camera.While this article adopts pure hardware structure and the system on a chip is agood way to solve the static gesture recognition, dynamic hand gesture recognitionand identification of the track problem, by replacing recognition module. But there issome ways from sign language recognition system, which is combined with thosethree directions. But it is very hard. It needs us to probe and research constantly. Toensures ourselves a bright future on the Human ComPuter Inieraetion(HCI).