Image Sequence-based Target Pose Estimation Method And Application For Unmanned Aerial Vehicle

Accurately perceiving the spatial states of itself or tracking targets is a fundamental work for unmanned aerial vehicles(UAVs)to perform tasks autonomously.In recent years,the research of target spatial localization has made remarkable achievements.As another type of spatial states,target attitude generally contains extra motion information which cannot be obtained from position.It is necessary to realize a joint estimation of target position and attitude for accurate target motion prediction.Therefore,studying the joint estimation of target spatial position and attitude becomes quite necessary for UAV intelligence improving.Based on sequence images,the study in this thesis concentrates on joint estimation of target position and attitude in UAV applications using generalized feature and deep learning feature of specified targets,respectively,with reliance on neural network technology to learn the priors of targets from data.The main works and achievements are summarized as follows:(1)A theoretical framework of target pose estimation based on sequence image was constructed.Firstly,the basic connotation and challenges of vision-based target pose estimation were analyzed,and then,we built the theoretical framework for sequence images-based joint estimation of target spatial position and attitude and proposed the target pose estimation scheme based on filter and recursion.Through analyzing and clarifying the principles of target pose estimation from traditional spatial geometry and new neural network,a new study approach for the vision-based target pose estimation problem was provided.(2)A filtering-based generalized feature-driven target pose estimation model was established.Based on the geometric constraints of target,the prediction equation and measurement equation of the target pose filter model were established and a theorical proof of the optimality of the filter model was given.This model made full use of the time domain correlation of target in sequence images,and had clear physical meaning and strong interpretability.In addition,its rigorous derivation process laid the foundation for accurate pose estimation results.Compared with classical method,the simulation results showed that the proposed pose filter not only increased the running speed by about 1 time,but also reduced the error of estimated pose by about 38%.(3)A multi-resolution detection method for anchors of target was proposed.According to the observation requirement of the generalized features-based target pose estimation model,the anchors were designed as generalized geometric features of the target and their configuration criterion was provided.To accurately detect the target anchors,we proposed a multi-resolution anchor detection architecture based on region partition.Through simulations,the multi-resolution anchor detection algorithm achieved an anchor detection accuracy of 98.7%,which was about 5% higher than that of the traditional single-channel anchor detection framework.(4)A deep feature-driven target pose end-to-end estimation model was developed.The neural network was employed to design a target pose end-to-end estimation model.Considering the visual feature extraction ability of the convolutional neural network and the historical state memory ability of the cyclic neural network,these two kinds of networks were combined to construct a target pose end-to-end estimation network.This network realized a high degree of integration of pose estimation procedure and reduced the cost of parameter debugging of the algorithm.The simulation test results verified the strong adaptability of the end-to-end network to different application scenarios.Compared with the classical end-to-end network,our network improved the position and attitude estimation accuracy by about 24% and 21%,respectively,without efficiency decreasing.(5)A calibration and optimization algorithm was designed for estimating conversion parameters of the developed end-to-end deep learning model.Since accurate conversion parameters is necessary for the measurement of the target pose end-to-end estimation model,a camera-gimbal-odometry extrinsic parameter optimization method was designed to obtain precious conversion parameters.Using this method,the parameters could be estimated in offline mode with manual assistance or online autonomy mode.This promotes its adaptability to different application scenarios.Simulations verified the great convergence of the parameter optimization method.Besides,using the optimized parameters,the average reconstruction error was less than3 m,which was only 2.6% of the distance between target and camera.It showed that the optimized parameters were with high accuracy.(6)A research and verification prototype of target pose estimation was conducted to demonstrate and verify the proposed target pose estimation algorithms with specified scenarios.According to ground vision-based UAV landing state monitoring and onboard vision-based ground target tracking scenario,real verification environments were constructed.The real experiments were conducted to verify the performances of proposed target pose estimation methods by comparing with classical pose estimation method.For the generalized feature-based method,the results showed that its average time consumption of is reduced by 90 ms,and its pose estimation accuracy is improved by 32%;and accordingly,the depth feature-based method achieved an improvement in the accuracy of 12% and 16% in position estimation and attitude estimation,respectively.All the above works focus on the method and application research of sequence image-based target pose estimation.This study lays a foundation for the deep integration of artificial intelligence technology and unmanned aerial systems.