Research On Monocular Textureless 3D Object Tracking

Estimating the pose of specific objects and tracking its pose over time is a sig-nificant task for many applications of computer vision.For example,in order to render or superimpose virtual information over a specific 3D object,augmented reality(AR)system needs to know the pose of target object;in order to grasp a specific 3D object,robotic system also requires tracking the pose of target object.3D object tracking is an important mean to acquire the pose of a 3D object over time.Many technologies have been proposed to achieve 3D object tracking:Me-chanical tracking system is accurate but limits the users in a constrained working space.Other hardware tracking systems with magnetic,inertial and ultrasonic sensors are sensitive to distortions in the environment and tend to be invasive.Computer vision-based 3D object tracking is a non-invasive,accurate and low-cost solution,with increasing computing power of mobile devices and the wide spread of portable wearable devices,3D object,tracking under monocular color cameras is highly valued in manufacturing,medical,education,entertainment industries,etc.Monocular 3D object tracking aims to continuously estimate the pose of the object relative to the camera in temporal sequences.Although researchers have been working tirelessly on 3D object tracking and proposed many methods in recent decades,3D object tracking is still a challenging research problem.Only color or gray information of the environment could be used,3D object tracking under monocular color cameras is especially difficult.These difficulties are mainly derived from multiple intractable factors including textureless objects,complex backgrounds,occlusions,fast motions,etc.Textureless objects lack of discrimi-native features,and clutter backgrounds could disturb the pose optimization of 3D object tracking.Moreover,occlusions and fast motions are inherent difficul-ties for most computer vision tasks.Considering monocular color cameras axe common in daily life,such as cell phone cameras and surveillance cameras,and most artifacts are textureless,such as industrial components,refrigerators and air conditioners,this thesis focuses on the problem of monocular textureless 3D ob-ject tracking using color images.Firstly,we study the pose optimization in edge distance field for textureless 3D object tracking;secondly,we study the consis-tency of edge direction for pose validation in the process of textureless 3D object tracking;finally,we study the edge suppression based on statistic color models and implementation of real-time robust 3D object tracking prototype system.The main contributions of this thesis are as follows:1.Proposing a pose optimization method in edge distance field for textureless 3D object tracking.Instead of explicitly searching for 3D-2D correspondences as previous methods,which unavoidably generate individual outlier matches,the proposed method aims to minimize the holistic distance between the predicted object contour and the query image edges.The proposed method can directly solve 3D pose parameters in unsegmented edge distance field.The differentials of edge matching distance with respect to the pose parameters could be derived analytically,and the optimal 3D pose parameters could be estimated by standard gradient-based non-linear optimization techniques.To avoid being trapped in local minima and to deal with potential large inter-frame motions,a particle filtering process with a first order autoregressive state dynamics is exploited.Occlusions are handled by a robust estimator.The effectiveness of this method is demonstrated using comparative experiments on real image sequences with occlusions,large motions and cluttered backgrounds.2.Proposing a robust 3D object tracking method with direction-based pose validation and pose recovery.We first introduce a pose estimation method in edge distance field.This method is accurate with a good initialization,however,it is sensitive to occlusion and fast motion,thus often gets lost in real envi-ronments.To improve robustness,we exploit consistency of edge direction for validating the correctness of the estimated 3D pose,and further incorporate the validation scheme for robust estimation,non-local searching and failure recovery.The robust estimation adopts point-wise validation to reduce the effect of outlier,resulting in a direction-based robust estimator.The non-local searching is based on particle filter,with the pose validation for a faithful weighting of particles.The failure recovery is based on fast 2D detection,and estimates the recovered pose by searching for 3D-2D point correspondences,with the validation scheme to adaptively determine state transition.The experimental results demonstrates the effectiveness of this method in real environments.3.Proposing and implementing a real-time stable monocular textureless 3D object tracking prototype system.The system runs in one of three states:ini-tialization,tracking and relocation,and switches to another state by verifying the pose consistency score.The system projects the 3D geometric model to the image,and iteratively searches the optimal pose of the object by minimizing the distance between the projected contour and the edge of the image.The object pose of the first frame is preset manually,and the subsequent initial pose of each frame is obtained by applying motion model to the pose of previous frame.In order to recover the system from tracking failures caused by fast motions or if the object moves out of the camera’s field of view,the system exploits fast 2D object detection with online key frames as the templates.Meanwhile,to suppress the distortions from edges of backgrounds,an edge classifier based on statistic color information is proposed to eliminate the non-object edges.