Framework Design And Algorithm Research On Airborne Machine Vision

Airborne machine visual(MV)system is the essential subsystem for flight safety insurance and pilot vision extension.With the improvement of high computing capability machine vision modules and the fusion of multi-module information,airborne subsystems have developed into a core information hub for each aircraft.Thus the level of airborne MV technology marks the intelligence of modern aviation.The demand for airborne MV applications has grown dramatically,from the autopilot of unmanned aerial vehicle(UAV)to rapid target detection on warhead,from the auto cruise of power lines to the real-time imagery analyses in orbit,the single mode airborne MV system is difficult to meet the blowing application demands.It is an urgent task to design a new airborne MV framework capable of processing general MV tasks and merging multi-module information.Targeted at the bottleneck of limited feature depth and restricted application fields,this dissertation takes the advantage of human multi-layer vision model along with GPGPU units and distributed computational structure.Three airborne MV structures are proposed,including one major structure and two substructures.Based on the proposed structures,two crucial algorithms on airborne three dimensional(3D)features and semantic features are introduced.The main contribution of this dissertation is listed below:1.To achieve the adaptive computational resource and multi-module MV processing,this dissertation presents the Adaptive Asynchronous Airborne Computer Vision Processing Structure(A3MV).The A3 MV solve the bottleneck of computational capability for manufactured aircrafts when processing MV tasks in complicated scenarios.Through introducing GPGPU units and distributed computational structure,A3 MV allows airborne MV structures perform flexible and extensional computing tasks.Problems including subsystem computational cost redundancy,low level information fusion and large processing latency are taken into the consideration in designing the new structure.The A3 MV structure lays a foundation for the military and civil application.The proposal of A3 MV is a milestone for the future machine vision algorithms on airborne platform.2.In order to solve the existing problems in transmitting ultra-high-resolution imageries from airborne platforms to ground stations with limited bandwidth,airbornemulti-modal feature based visual tunnel(AMF-VT)is proposed.The AMF-VT structure introduces the saliency system structure in human vision interest mechanism,achieving real-time region extraction for video transferring based on limited digital communication bandwidth.During the experimental procedure,the information from IMU sensors are integrated into the system to accelerate the procedure.With limited area of super-pixel update,the computational cost of super-pixel extraction for high resolution airborne images are reduced.The AMF-VT is an ideal tool framework to accelerate the deployment of high level machine vision algorithms on airborne platforms.3.To meet the urgent need for recognizing massive objects in ultra-high resolution airborne images,Distributed Multi-layer Airborne Machine Vision Structure(DMA-MV)is proposed in this dissertation.Traditional object recognition method deploys the object recognition procedure frame-widely,bringing unsustainable computational cost for embedded systems.The proposed DMA-MV structure utilizes assist information from additional sensors to accelerate the high value region identification.With scale limited high value region,the computational cost of object recognition procedure is target oriented,contributing to the high object recognition recall rate and precision rate with sustainable computational cost for embedded systems.The DMA-MV structure is a theoretical fulcrum for the design of future algorithms.4.For the three-dimensional information extraction method based on airborne platforms,Spatial-temporal Shadow Aggregation based Structure From Motion Method(SA-SFM)is proposed.Based on the temporal-spatial aggregation of discrete silhouette information extracted from shadow,the proposed SA-SFM algorithm can of extracting major joint 3D positions.Based on the 3D joint positions,the skeleton of human can be recovered to describe the human activity.Additionally,the positions of UAV rotors are extractable as well,allowing the UAV status to be accessible.The proposed method distinguished with the traditional 3D reconstruction methods for UAV platforms in two aspects including low power consumption and no special sensor requirement,allowing the 3D information extraction based on micro UAVs equipped with normal monocular cameras.The SA-SFM algorithm provides the port for airborne three-dimensional target structure analyses,possessing high application value on military reconnaissance and human-machine interaction.5.In order to deploy the high-level verbal feature based region understanding structure in airborne platform,a top-down region segmentation procedure and bottom-up region feature extraction procedure must be carried out.However,normal bottom-up feature extraction method requires complicated training database and longtime of feature extraction training procedure.To solve this problem and adopt the verbal feature on airborne platform,a new algorithm based on affordance feature is introduced.Through utilizing the tight relationship between object activity and regional affordance feature,the affordance feature for different regions can be automatically marked by temporal aggregated object activities.Based on the proposed algorithm,regional segmentation and understanding for compound region can be achieved with relatively high precision.The proposed region understanding structure validate the feasibility of deploying high-level deep learning algorithms on airborne machine vision platforms,contributing to future engineering application of high level airborne machine vision algorithms.