Unsupervised Generative Adversarial Learning For 3D Scene Flow From Stereo Images

3D scene flow characterizes how the points at the current time flow to the next time in the 3D Euclidean space,which possesses the capacity to infer autonomously the non-rigid motion of all objects in the scene.Previously,image-based scene flow estimation networks separately estimated optical flow and disparity,which segmented the three-dimensional scene flow into two sub-tasks.They were unable to directly estimate the three-dimensional motion vectors of each point in the real three-dimensional physical space.Self-supervised learning of 3D scene flow on point clouds also still has challenges: Sparsity of real-world Li DAR point clouds and point cloud distortion caused by dynamic objects;There is a gap between the synthesized dataset and the real-world dataset;The performance of existing self-supervised learning networks is poor.To address these challenges,This thesis focuses on the research of unsupervised learning based on image-based 3D scene flow,and the details of the study are as follows:(1)This thesis proposes a method to directly estimate 3D scene flow by generating dense depth maps and obtaining explicit 3D coordinates.This approach addresses the limitation of existing methods that cannot recover 3D scene flow from the image space and effectively avoids interference caused by motion distortion of point clouds.To improve the robustness of network performance,a pseudo-Li DAR point cloud neighborhood statistical analysis filter module is proposed to optimize the generated point clouds.The designed network is trained from the camera signals in real-world autonomous driving scenarios,allowing it to learn knowledge that is not bound by synthetic data and thus addressing the problem of domain mismatch between synthetic and real-world datasets.(2)This thesis addresses the low accuracy in estimating scene flow caused by existing algorithms not subject to rigid constraints.To overcome this issue,a mask-weighted warping layer is designed to divide visible points into dynamic,static,and occluded components.Different weights are assigned to points in different states during the refinement process of scene flow to obtain more accurate predicted point clouds.(3)To enhance the unsupervised learning capability of the network for scene flow estimation,this thesis proposes to improve it from three perspectives.First,a disparity consistency loss is introduced to constrain the temporal consistency of dense pixel depth.Second,rich odometry labels are used to constrain rigid flow in scene flow,and the proposed network is trained with real-world Li DAR data.Third,generative adversarial ideas are introduced to improve unsupervised learning capability.A generator and discriminator are designed to complete unsupervised adversarial training.(4)This thesis implements a robust unsupervised 3D scene flow learning framework on pseudo-Li DAR point clouds to address the poor generalization of current 3D scene flow estimation networks.Experimental results show that the proposed method outperforms the method that learns scene flow from the synthetic dataset Flying Things3 D.Evaluation results on the popular sf KITTI dataset show that the proposed method improves performance by about 45% compared to the baseline network.The generalization of the model is also verified on three datasets,Argoverse,nu Scenes,and lidar KITTI.This thesis contains 28 figures,11 tables and 135 references.