Design Of Embedded Human-computer Interaction System Based On Face And Eye Trackin

Gaze tracking technology is one of the main research areas in the field of computer vision,and new human-computer interaction based on face vision tracking has become an important research topic in the field of multichannel natural interaction.The human-computer interaction method through gaze tracking has changed the traditional human-computer interaction approach,increasing the possibilities of human-computer interaction and providing users with a novel,natural and efficient human-computer interaction experience,and can also be applied in some areas of rehabilitation medicine.This paper introduces the application of gaze tracking technology in different fields and designs an embedded human-computer interaction system with face gaze tracking for cursor control,including: hardware circuit design,embedded Linux system porting,QT graphics library porting,OpenCV image vision library porting,Dilb machine learning library porting,application design and host computer software design.This design uses the cursor control device to process the captured face images,then sends control commands to the PC through the serial port,and finally the software of the upper computer cooperates in order to realise the control of the cursor.The cursor control device adopts the I.MX6 ULL chip with ARM Cortex-A7 architecture as the core processor,captures the face image using the binocular camera device with USB interface,and completes the circuit design of each functional module,while building the embedded software platform.The application is developed by using ERT algorithm for face feature point detection,judging the offset angle of head posture by binocular stereo vision principle and stereo geometry principle,while intercepting the smallest rectangle of the eye,and after image processing and operation,getting the coordinates of the iris centre of gravity point,and according to its mapping relationship with the gaze point on the PC monitor,determining the gaze point in the resting state of the head,and then combining the offset angle of head posture to locate the actual gaze point;In addition,it is possible to determine whether a mouse click event is triggered based on the change in the geometric relationship of the mouth.The host software in this design is used to verify the functionality of the human-computer interaction system to achieve cursor movement and click control of the PC desktop cursor by the cursor control device.After the experimental test,when the control device of this system is working,the comprehensive effective recognition rate of face feature point detection is about 96.3% on average,the accuracy of gaze tracking reaches within 2.4°,and the average frame rate of video recognition of the system reaches 11 FPS,which meets the basic requirements for cursor control and can effectively carry out human-computer interaction through gaze tracking.