Research On Eye Gaze Targeting Technology For Unmanned Weapon

In order to adapt to the informationized and unmanned development trend of the future war,unmanned weapon systems including unmanned combat platform,remote control weapon station,unmanned duty weapon,top mounted weapon and etc.emerge as the times require.As the competing research hotspot in countries around the world,it becomes the important direction for future development of weapon equipment.This paper works on the eye gaze targeting technology for unmanned weapon.A head-mounted man machine integration gaze tracking and targeting operation system based on head-mounted display and eye movement tracking is constructed.Head-mounted display,eye movement tracking device,operater and weapon pan-tilt are coupled tightly.Eye gaze is tracked by detecting eyeball movement.The fixation information when the operater is gazing at a particular target is captured.The following up relationship between operater's eyeball movement and weapon target tracking and targeting is established.The weapon pan-tilt is drived to track and target following the operater's sight line.“See where attack where,seeing is death”is realized.It makes great sense on realizing unmanned weapon efficient and precise strike and reducing the operater's the burden.The main research work of this paper is as follows:Research on eye movement mechanism and design of eye gaze tracking system.Eye structure and motion mechanism including the eye physiological structure and the vision principle,oculomotor control,basic model of eye movement,eye model and etc.are studied.Head-mounted gaze tracking system based on single camera and near-infrared LED array is constructed on the theoretical basis of eye model and movement mechanism.Research on human eye detection and location technology.In this paper we propose a hybrid weighted feature(H-LDP~2)based on Haar-like and 2nd order LDP feature for human eye detection.A certain number of positive and negative human eye sample images are acquired for the extraction of H-LDP~2 feature.In order to acquire cascade eye classifier for human eye detection and location,Adaboost algorithm is utilized to train hybrid feature model.Before feature extraction,Gabor operator is used to filter the human eye target image in multi scale and multi direction to extract eye characteristics of different scales and directions.Research on detection and localization technology of pupil and glint.A precise and stable pupil and glint detection method is proposed in this paper.For pupil detection,a circular ring rays location(CRRL)method is proposed to detect pupil boundary points.Rough pupil center is located and initial pupil radius is calculated.A circular ring area including pupil edge inside is determined according to rough pupil center and initial pupil radius.A series of rays are shot from inner to outer ring to collect pupil boundary points by calculating gradient magnitude.An improved total least squares is proposed to fit collected pupil boundary points.For glint detection,rough regional location is determined by thresholding and connected region detection.Gaussian function deformation and improved total least squares is combined together to calculated its center position.Research on gaze direction estimation technology.A 2D gaze estimation method based on pupil-glint(P-CR)vector is proposed.CRRL and Gaussian fitting is utilized for pupil and glint detection respectively,according to which the P-CR vector is calculated.A mapping function is established according to the corresponding relationship between P-CR vectors and actual gaze calibration points.For the purpose of estimating gaze direction,an improved artificial neural network(DLSR-ANN)based on direct least squares regression is proposed to solve the mapping function.Finally,error compensation is implemented to further enhance accuracy of gaze estimation.Weapon servo pan-tilt following up control technology research.In order to realize the fast response and the transfer of the servo motor,a three closed loop control model of the motor based on fuzzy PI is established.Calibration of eye gaze tracking system is conducted.Gaze direction is estimated and gaze point coordinate is marked on the head-mounted monitor screen.The reconnoitering scene of weapon targeting camera is diaplayed on the head-mounted monitor.During the process of observing and targeting,gaze direction is converted to the control parameters of the unmanned weapon servo system.Motors of servo system are drived to move until the cross line of weapon targeting camera and the target is overlapped.The true sense of“See where attack where,seeing is death”is realized.In order to verify the gaze targeting technology proposed in this paper,NVIDIA Jetson TX1 development board is chosen as the data processing unit.Linux C++is utilized to write the host computer program algorithm including human eye detection and localization,pupil and glint detection and localization,mapping function calibration,gaze direction estimation,servo pan-tilt following up control and etc.Experiment is carried out on weapon pan-tilt principle prototype.The experimental results show that the gaze targeting technology studied in this paper can fulfil the requirements of accuracy,stability and real-time performance of the gaze targeting operation for unmanned weapon.