Identification Of Aviator Operation In Cockpit Based On Computer Vision

With the development of aviation technology,the proportion of flight accidents caused by aircraft equipment has been greatly reduced,while human factors are gradually becoming an important factor restricting the development of aviation safety.As an important component of Transport aircraft airworthiness standards,pilot work load is a significant content for the human factors research in cockpit.The purpose of this paper is to use computer vision to detect and track the pilot's gestures,and identify the type of operation,the beginning and duration of operation intelligently,and support the pilot work load research and further research on human factors in the cockpit.The main contents of this paper are as followed.The common 3D reconstruction techniques are discussed.And infrared depth camera is used to obtain the depth information and achieve 3D reconstruction on the cockpit.A method of hand tracking algorithm is proposed based on the region growing technique and information about the 3D point cloud.This handtracking method can avoid the drawbacks such as facing the camera in front and restricting the area of hand movement.And experiments are designed to validate the effectiveness of the tracking algorithm.According to the standard operating manual,three kinds of basic operations are presented.Based on the operational segment,a method for identifying the operation of aviators is proposed with the hand tracking information.Then the factors that affect the recognition effect are analyzed.In accordance with the standard operation process,we designed the manual flight experiment,and analyzed the experimental data.The experimental results show that the proposed scheme can effectively identify the type of pilot operation,and calculate the beginning,the end and the duration of the operation.