The Design And Implementation Of Binocular Vision Based3D Writing System

Real-time human-computer interaction technique plays an important role in both theory and application of telerobots, virtual reality and telemedicine. And the3D real-time human-computer interaction based on hand gesture technique is a new interactive mode, which is closer to nature, convenient and intuitive. Binocular stereo vision is an important form of machine vision, with many advantages such as high efficiency, suitable precision, simple structure and low cost. Via binocular stereo vision, image can be acquired immediately when recognizing gestures, thus the measurement can be performed efficiently.This thesis focuses on establishing a3D handwriting system. This system is designed for restoring the user’s fingertips trace in3D graphical environment and displaying the writing contents in real-time, based on binocular vision and gesture recognition technologies.The primary contents of this thesis are:Firstly, the recognition and acquisition of the fingertip position. Use skin color segmentation and fingertip detection algorithm to obtain the user’s fingertip position. Implement and compare some kinds of online skin detection algorithm, and ultimately select the best one. By comparing and analyzing some single fingertip algorithms based on skeleton endpoint searching, we select a single fingertip detection algorithm which is based on the distance to the gravity center. Experimental results show that this algorithm has a good robustness.Secondly, the3D reconstruction of the fingertip position. Use the binocular vision technology to get the position information of fingertip in3D space by calibrating with cameras and3D reconstructing from the2D fingertip position gained in last step. The camera calibration uses the classic cheeseboard and OpenCV calibration function to obtain the origin point and orientation in the world coordinate system.Thirdly, the prediction and smoothing of the fingertip trajectory. Smooth fingertip trajectory and calculate3D fingertip position through the Kalman filter algorithm. The predicted3D position will be projected to the image planes of two cameras, and then the ROI foreground segmentation will be set.Finally, design and implement a prototype of3D handwriting system based on above key algorithms.The experimental result shows that the3D handwriting system can detect and display the3D fingertip accurately. With the popularization of3D camera on mobile terminal and wearable computer, the system has a broad application prospect, such as3D mouse and virtual character manipulation in3D games. In addition, with combination of language, attention, lips and other interactive ways, we can complement a multi-channel, multi-modal human-computer interaction. It will further promote the development of virtual reality technology and be of great value in research.