Development Of Sign Language Gesture Recognition System Based On RealSense

With the rapid development of contemporary computer vision technology,human-computer interaction has become a new trend in all walks of life.Sign language gestures are the most primitive way of communication,and the interaction of recognition is of high research value.Integration into normal social life is of great significance.At present,the research on sign language gesture recognition mainly uses RGB cameras and data gloves.However,the cost of data gloves is relatively high,it is inconvenient to wear,and it is difficult to apply it in life.With the emergence of 3D cameras such as Kinect and RealSense,visual sign language gesture recognition has greater development prospects,and has become one of the hot researches today.This article uses Intel’s RealSense3D camera to collect sign language gesture video images,and uses depth information and skin color detection to merge the hand segmentation method for related acquisition processing.This method can effectively reduce the effects of complex background and lighting conditions.Then the static sign language gesture recognition and the dynamic sign language gesture recognition were studied respectively.In the study of static sign language gesture recognition,corresponding image preprocessing was performed on the collected sign language gesture images,including median filtering and gesture edge contour extraction.Median filtering uses a 9 * 9 window template for filtering and denoising.The Canny operator is used to extract the edge contour of sign language gestures.Then a Hu moment feature extraction method combining roundness,rectangularity and aspect ratio is adopted,and the static sign language gestures of different features are classified and recognized by SVM and BP classification and recognition algorithms.As a result,the SVM algorithm is found in this paper Medium recognition works best.In the study of dynamic sign language gesture recognition,the improved KCF algorithm combined with Kalman filtering is applied to hand tracking,and then the feature method of combining hand shape and motion trajectory is used.The feature extraction of hand shape uses the method of static sign language gesture,and then combined The improved DTW algorithm is used for dynamic sign language gesture recognition.Finally,the signlanguage gesture recognition system is designed and implemented using VS2017,Open CV,and Qt,and its function is tested experimentally.Through experimental verification and system testing,the system completes the recognition of numbers 1-10 and 30 sign language gesture words.The average recognition rate of the experiment is 90.1%,and the average time is 0.23 s,which meets the real-time effect requirements.