The Research Of Scene Flow Estimation And Augmented Reality On Monocular Traffic Video

The core research in the field of computer vision is to get structure of camera's imaging environment and motion information of object in scene.The algorithm of motion to structure in 2d scene and the solution technique of multicast scene flow are all based on motion information of image and geometric relationship.As a continuous image collection,video provides the image's color information,geometry and time constraints for existing image algorithms which makes solving scene structure and motion information in video become impossible.Meanwhile,with the development of driverless technology,the scene understanding and motion recognition based on camera is another way provides the driving guide except to be provided by GPS and radar.The 3d motion information scene flow which improved from 2d video construction provides the basis of scene structure understating and object recognition,and become one of the focus in field of image research.By studying monocular pinhole camera model,camera calibration parameters can be obtained,after combing the coordinate transformation in imaging process,the correspondence between points in world space and image space pixels can be inferred.This article focuses on current demand of automated vehicle,combines traffic monocular video features and camera imaging model,gets following research progresses:Due to current autonomous technology requires a lot of space situational awareness data,including the movement of objects.This article proposes a monocular dynamic camera scene flow estimation algorithm by combining single camera calibration matrix and extrinsic papameters,which provides point clouds positions and motion fields for space motion segmentation and collision detecting.The algorithm firstly presents three important assumptions based on space-time consistency of video,formulates them as energy functional items.Secondly,presents an isotropic and an anisotropic operator as energy functional constraint term,according to the scene flow and depth distribution.In addition,for overcoming the occlusion brought from movement,an occlusion mask algorithm is proposed;after this,the functional can be rewritten as a non-linear iterative formula,by using Euler-Lagrange equation,then the scene flow estimation results can be obtained two iterations of Gaussian pyramid.Finally,we do experiments on multiple datasets,and the results are compared with the current common binocular scene flow algorithm,to access the effectiveness of this article algorithm,and prove its greater flexibility.Currently,researches on traffic scene become popular,but the highly variability,different weather conditions make collecting traffic datasets with diversified driving environment very hard.For this reason,this article studies camera model,designs and realizes an augmented reality data generation system,which makes existing traffic monocular video and calibration parameters as input,adds virtual models to the scene following the principle of geometric consistency,generates a variety of traffic scene video datasets.The system using 3D game engine as rendering tool,using image processing library as a video loaded tool and using WPF as interface tool.The process of this system can be explained as:firstly,transforming calibration parameters into engine's camera projection model,which ensures the geometric consistency between real and virtual world;secondly,making Unity Player encapsulated in the window component and then embedded in WPF widget;implementing communication between interface process and rendering process by message mechanism;at last,combing the simple linear interpolation and spherical linear interpolation into a keyframe interpolation framework,which makes users set position and rotation of models easily.Augmented reality system also simulates different weather condition,for improving diversity of generated traffic video.