Vision-based Ground Moving Object Tracking For MAVs

In recent years,vision-based ground moving object tracking has become a very active field of research for macro aerial vehicles(MAVs),which mainly contains two problems:the pose estimation for MAVs and robust object tracking based on image sequences.Accurate pose estimation is the prerequisite to achieve autonomous flights and realize robust moving object tracking for MAVs.Because of low cost and natural complementary characteristics,the monocular camera and the inertial measurement unit can be combined to provide an attractive solution for MAVs' pose estimation.The monocular camera is an exteroceptive sensor that can be used to measure scene appearance and recover its geometry with an unknown metric scale,and an inertial measurement unit is a proprioceptive sensor that makes the metric scale of the monocular vision system observable and provides robust and accurate short-term motion estimations.However,although good results have been obtained by the visual-inertial system,it is still a challenging problem about how to combine visual and inertial measurements for robust and accurate pose estimations with the limited computing power.As a major and challenging research topic in the computer vision community,object tracking has been researched for several decades,and the great achievements have been achieved.However,there are still some unavoidable problems for MAVs to be solved in object tracking.For example,how to realize real-time object tracking with the limited computational capacity of an on-board computer;fast maneuvering of MAVs;changes of the scene illumination and the object's appearance;frequent scale changes;fast rotation of the tracked object in the image;etc..In terms of the two problems mentioned above,the following research is performed and finished:(1)A tightly coupled visual-inertial pose estimation system based on the sliding window approach is proposed for MAVs.Firstly,Lie Groups and Lie Algebras are used to deduce our proposed algorithm for visual-inertial pose estimation in detail.A visual-inertial initialization approach is proposed to recover the metric scale,velocity,gravity vector,gyroscope bias,acceleration bias and the height above ground.The initial values are used to bootstrap our proposed tightly coupled visual-inertial pose estimation system,and robust and accurate pose estimation can be realized for MAVs.In practical applications,a limited number of sliding windows of the most recent visual and inertial measurements are used,thus good real-time performance can be achieved.Simultaneously,historical measurements in the sliding windows can improve the accuracy of pose estimation and make the estimated trajectory more smooth.(2)High tracking frame rates have been achieved based on traditional tracking methods which however would fail due to drifts of the object template or model,especially when the object disappears from the camera's field of view.To deal with it,this thesis proposes a framework of parallel tracking and detection(PTAD)for unknown object tracking.The object tracking framework is split into two separate tasks——tracking and detection which can be processed in two different threads respectively.This framework has several advantages.On the one hand,an independent detector can recover or correct a tracker when it fails,or drifts are large.On the other hand,the combination of the tracking trajectory generated by the tracker and the object's location obtained by the detector can provide unlabeled training samples for the detector with temporal-spatial structure constraints.The experiments are performed on the TLD datasets,and the experimental results validate the effectiveness of the proposed PTAD framework.(3)A robust object tracking algorithm with adaptive scale and rotation estimation is proposed to deal with the challenges of scale and rotation changes during the object tracking.Firstly,a robust scale and rotation estimation method is proposed based on the Fourier-Mellin transform and the kernelized correlation filter to deal with scale changes and rotation motions of the object during object tracking.Then a weighted object searching method based on the histogram and the variance is proposed to deal with that the trackers may fail in the object tracking(Due to long-term semi-occlusions or full occlusions,etc.).When the tracked object is lost,the object can be re-located in the image using the proposed searching method,so the tracker can be recovered from failures.Moreover,this thesis also trains two other kernelized correlation filters to estimate the object's translation and the confidence of tracking results.The online object tracking benchmark(OTB)datasets are used to perform the experiments.The experimental results validate the effectiveness and superiority of the proposed tracking algorithm.(4)When the poses of MAVs and the poistions of the tracked object in the image are determined and the object is constrained to move on the ground plane,a linear estimation method is proposed to effectively obtain three-dimensional positions of the tracked ground object.