Design And Implementation Of UAV Target Tracking Algorithm Based On Event Camera

Nowadays,unmanned aerial vehicles have played an irreplaceable role in various fields,and the target tracking task is an extremely important function for unmanned aerial vehicles,such as the mobile follow-up function of civilian unmanned aerial vehicles,the function of fire fighting unmanned aerial vehicles to track the trend of mountain fires,etc.Therefore,it is particularly important to develop an effective target tracking algorithm suitable for UAV application scenarios.Previous single target tracking algorithms can be roughly divided into two categories,one is traditional methods,such as single target tracking algorithm based on correlation filtering;The other is deep learning methods,such as single target tracking algorithm based on twin neural network and fusion tracking algorithm based on visible light and infrared.But when the drone only carries RGB cameras,Unable to effectively adapt to more complex UAV tracking scenes,the existing methods are faced with the following problems: first,due to resonance caused by blade pulling force and centrifugal force and the influence of external wind force,the images taken by the camera will appear motion blur,and the computer vision algorithm cannot be effectively used for single target tracking;Second,the working environment of unmanned aerial vehicles is relatively complex;Traditional cameras combined with vision algorithms have low robustness in complex illumination scenes,which makes unmanned aerial vehicles unable to effectively track objects at night or in overexposed scenes;Third,in the scenario of UAV edge calculation,the embedded model is required to have high calculation speed,and the model without accelerated optimization is difficult to adapt to the scenario of edge calculation.In order to solve the above problems of target tracking from the perspective of UAV,this paper uses UAV to carry DAVIS event camera for target tracking.Compared with traditional cameras,event cameras have three characteristics: high time resolution,high dynamic range and low power consumption.They can not only transmit gray images(APS images),but also transmit event information.Due to the special imaging mechanism of DAVIS itself,imaging blur will not be generated due to the rapid movement of the target object,DAVIS can effectively adapt to the jitter scene of unmanned aerial vehicle,and can collect edge information of the object to be tracked more effectively than RGB camera in complex scenes such as special illumination.According to these characteristics,this paper designs a dual-modal fusion tracking network based on event and gray image,which effectively uses the edge information of event domain data and the texture information of APS domain data,and combines the two modal information for target tracking.In order to better train the dual-modal fusion tracking network of events and gray images,this paper uses the motion capture system Vicon to make its own target tracking data set from the perspective of unmanned aerial vehicles: Event-APS28.In addition,in order to adapt to the edge calculation scenario of UAV,TensorRT technology is used to optimize and accelerate the trained bimodal fusion tracking network model,and finally the optimized model is deployed in Jetson TX2 to realize edge AI calculation on UAV side.Experiments show that compared with other target tracking algorithms,the method proposed in this paper can be more effectively applied to the target tracking task in the UAV scene,and the tracking strategy is transplanted to the UAV side through optimization methods such as TensorRT to realize the rapid tracking of the UAV in the real scene.