Robust Model Fitting For Multi-structure Data And Its Applications

Robust model fitting is a critical research topic in computer vision,and it aims to estimate meaningful model instances from multi-structure data contaminated with outliers and noise.It has been widely employed in many practical vision applications,such as lane detection,motion segmentation,image registration and 3D reconstruction,etc.With the rapid development of the artificial intelligence,the data processed by artificial intelligence systems inevitably contain outliers or noise generated by sensors,environment,or human factors,which brings huge challenges to robust model fitting.In addition,the observed data may contain more than one model instance and unbalanced numbers of inliers belonging to different model instances,which further cause that traditional model fitting methods are still far from being satisfactory to deal with real-world problems in terms of fitting accuracy and speed.In this dissertation,to address three key issues(i.e.,unbalanced data,invalid model hypotheses,and a high proportion of outliers),four robust model fitting methods have been proposed to improve fitting accuracy and computational efficiency for multi-structure data.The proposed methods aim to integrate the advantages of traditional model fitting methods and address the issues such as a high proportion of outliers,a large number of invalid model hypotheses,and unbalanced data,to segment multiple moving objects(i.e.,multi-structure data)with occlusion or appearance variations.The main contents and contributions of this dissertation are as follows:(1)Aiming at the problem of inlier unbalanced in multi-structure data,a hypergraph optimization based model fitting method(HOMF)is proposed.The number of inliers contained in each structure of multi-structure data is usually unbalanced,which may cause some minor structures that only contain a few inliers to be ignored.In order to overcome the above problems,a novel hypergraph optimization based model fitting(HOMF)method is proposed.Specifically,the proposed method starts from a simple initialization hypergraph,where each vertex of the hypergraph represents a data point,and each hyperedge represents a model hypothesis.An iterative hyperedge optimization algorithm(IHO)is then developed to optimize the hyperedge of the initialized hypergraph to reduce the computational complexity of the hypergraph.Next,an adaptive inlier noise scale estimation algorithm(AIE)is proposed to separate significant vertices and insignificant vertices,where insignificant vertices are used to guide subsequent sampling of other structures.Finally,the optimized hypergraph is segmented by using a spectral clustering algorithm to estimate model instances.Experimental results show that the proposed HOMF method can effectively reduce the hypergraph computational complexity,and it can obtain good segmentation results.For the segmentation accuracy,HOMF is about 50.4%and 40.6%higher than the popular fitting methods T-Linkage and RPA,respectively.For the computational efficiency,HOMF is about 110.9 times and 76.1 times faster than T-Linkage and RPA,respectively.(2)Aiming at the problem of low-quality model hypotheses generated from multistructure data,a nonnegative matrix underapproximation and pruning techniques based model fitting method(NPMF)is proposed.Since multi-structure data usually contain a large number of outliers and pseudo-outliers,model fitting methods need to generate a great number of model hypotheses through random sampling to improve the probability of hitting real model instances.However,outliers and the generated invalid model hypotheses may affect the accuracy of model fitting.Therefore,a mismatch pruning algorithm is proposed to alleviate the influence of outliers by using a mismatch removal technique.Then,a model hypothesis pruning algorithm is introduced to prune insignificant model hypotheses,while retaining significant model hypotheses by using the weighting scores of model hypotheses,to construct a high-quality nonnegative preference matrix.Finally,both the spatial constraint and the sparsity constraint are combined to solve the optimization problem of nonnegative matrix underapproximation,to adaptively extract more specific information and coherent features,by which the robustness of NPMF can be effectively improved.Experimental results show that NPMF can effectively estimate the number and parameters of model instances in multi-structure data containing a large number of outliers,and NPMF obtains better segmentation performance compared with several representative fitting methods.For the segmentation accuracy,NPMF is about 197.2%and 47.7%higher than the representative fitting methods T-Linkage and RS-NMU,respectively.For the computational efficiency,NPMF is about 2.3 times and 1.9 times faster than T-Linkage and RS-NMU,respectively.(3)Aiming at the problem of high proportion of outliers in multi-structured data,a hierarchical message propagation based model fitting method(HRMP)is proposed.Multi-structure data in real scenes usually contain a high proportion of outliers,which leads to generate inaccurate model hypothyses,resulting in a decrease in fitting accuracy.Therefore,firstly,a hierarchical representation is constructed by taking advantages of both the consensus analysis and the preference analysis.Then,a novel hierarchical message propagation algorithm(HMP)is proposed to prune the two-layer nodes of the hierarchical representation(corresponding to data points and model hypotheses,respectively),thereby reducing the influence of high-proportion outliers and invalid model hypotheses on the algorithm.Moreover,an improved affinity propagation algorithm(IAP)is proposed to cluster the remaining data points to estimate the number and parameters of model instances.Experimental results show that the proposed method significantly outperforms several state-of-the-art model fitting methods in terms of fitting accuracy and speed.For the segmentation accuracy,HRMP is about 84.6%and 67.0%higher than the representative fitting methods T-Linkage and RansaCov,respectively.For the computational efficiency,HRMP is about 22.6 times and 31.0 times faster than T-Linkage and RansaCov,respectively.(4)Aiming at the problems of occlusion or appearance variations of moving objects,an attention induced heterogeneous model based model fitting method(HMFMS)is proposed.In real scenes,appearance and illumination variations,and occlusion of moving objects will cause some information to be missing in video sequences.To overcome the aforementioned problems,an attention-guided heterogeneous model construction algorithm(ATT)is proposed to construct high-quality accumulated correlation matrices,by evaluating the quality of heterogeneous model hypotheses,based on the density estimation technique.Then,an adaptive heterogeneous model refinement algorithm(REF)is proposed to construct sparse affinity matrices from the accumulated correlation matrices by applying information theory,where the values of correlations between different objects are effectively suppressed.Finally,a heterogeneous model segmentation algorithm(SEG)is proposed to segment moving objects based on sparse affinity matrices.Experimental results show that HMFMS achieves superior performance on four popular and challenging datasets,compared with several representative subspace based and model fitting based methods.For the segmentation accuracy,HMFMS is about 639.7%and 76.4%higher than the representative fitting methods SSC and SUBSET,respectively.For the computational efficiency,HMFMS is about 1.5 times faster than SUBSET.