Visual-Inertial Location System With Point And Line Features For Blade Inspection UAV

With the development of industrial detection technology with UAV,many wind farms gradually apply multi rotor UAV equipped with digital camera to photograph the surface damage of large-scale wind turbines.However,the current UAV inspection is mainly realized by planning the waypoint in advance and combining with the control of ground operators,which reduces the efficiency of inspection.At the same time,the strong electromagnetic interference in the wind farm often makes the GPS sensor of UAV invalid when the UAV is close to wind turbine for inspection work.This seriously affects the precise location of UAV.In order to improve the efficiency of UAV and the robustness of location system,this paper studies and develops a visual-inertial location system with point and line features.Specifically,the following research has been carried out:(1)Considering The influence of outdoor environment light on UAV vision,the simultaneous localization and mapping(SLAM)based on stereo vision is adopted to realize the location.According to the low-textured characteristics of the wind turbine image captured during the UAV flight,the point feature and line feature are combined to achieve the data association of images and tracking task.In order to further improve the accuracy and efficiency of feature matching process,a series of filtering,merging and speed-up optimization are adopted for line feature extraction.(2)Aiming at the tracking failure of vision-only system when the UAV is in fast motion and pure rotation motion,the information of inertial measurement unit(IMU)which is easy to detect fast motion is introduced into the vision location system.The kinematic model of IMU is established,and the IMU constraints between image keyframes are calculated according to the theory of IMU pre-integration on manifold.(3)In order to realize the fusion of visual-inertial information through tight coupling,a non-linear optimization backend based on graph optimization is constructed.The rotation,translation,position of point and line landmark to be solved by the system are selected as state variables.The re-projection error of point feature and line feature as well as the measurement error of IMU are integrated into the graph optimization framework.The spatial lines which are not sensitive to the change of illumination are selected to parameterize the line features.Plücker expression of lines is selected in the re-projection,and Cayley expression is selected in the back-end optimization.The Jacobian matrixes which can be used in back-end optimization are also derived.(4)The outdoor location experiment is designed.Firstly,the hardware platform of six rotor UAV is built,and the sensors are synchronized and calibrated.Then the overall framework of the software system is planned.In the part of quantitative experiment,mobile robot platform which is easy to get the ground truth of location is selected to simulate the movement of UAV.In the wind farm,data sets including various kinds of situations are collected to quantitatively evaluate the location accuracy of the algorithm.The experimental results verify the accuracy and adaptability of the algorithm.Furthermore,the flight experiment is carried out for UAV,which qualitatively verifies the continuity and stability of the algorithm in the process of UAV inspection.It shows that this system effectively improves the location robustness of inspection UAV in wind farm.The mapping result in this paper illustrates that the system is able to get denser and more structured environment map by the addition of line features,which is of great significance for autonomous navigation in the following inspection.