Research On Moving Object Detection And Scene Reconstruction Based On Laser And Visual Information Fusion

As a core technique in the video surveillance and 3D map building fields,the moving object detection and scene reconstruction are the foundations of the real-time navigation,obstacle avoidance and path planning.In the meantime,as two important sub-problems of environment perception task,they are not only closely connected with the development of many fields,such as robot,unmanned aircraft,unmanned vehicles and body feeling game,but also are the part of the lives of human intelligence.The current research mainly based on single laser sensor or single vision sensor.It is difficult to meet the requirement of real-time multi task scenario due to the limitation of the visual field,the amount of data,the richness of the data,the real-time and the anti-jamming.Based on the mutual supplement and constraint of the laser information and the visual information,a real-time system for moving object detection and scene reconstruction is designed,and the two sub problems are studied in detail.The main research contents and innovations are as follows:Firstly,based on the multi-sensor hardware platform Multisense-SL,compared and analyzed the differences in performance between laser sensor and vision sensor.Then,demonstrated the rationality of the fusion of laser and vision in the dynamic scene perception.And based on the model of the message publishing and subscription on the software platform ROS,the whole fusion framework can be designed to carry out multiple real-time perception tasks simultaneously.Secondly,aiming at the problem of the error detection for uncovered background area,and the problem of the incomplete extraction of the 3D foreground,a novel fusion motion detection algorithm is proposed under the feature layer.The laser foreground points was firstly detected by visual background subtraction algorithm.Then regarded laser foreground points as the heuristic information,carry out the clustering by 2D image neighborhood searching under the constraints of 3D distance.In the end,the 2D image foreground and the 3D point cloud foreground can be get.While the foreground object is extracted completely,the detection error of the background exposure area is effectively suppressed.Thirdly,in order to solve the problem of the noise in the low-level fusion and the mismatch of the point cloud directly,which is caused by the direct registration based on the external calibration relationship between the sensors,related optimization strategies are proposed before the scene reconstruction.In the preprocessing stage,the field constraint,time constraint and occlusion constraint are proposed to effectively filter out mostly noise.In the integration optimization stage,based on point cloud segmentation and ICP algorithm a novel mismatching correction algorithm is proposed.Compared with the correction result of the whole registration,the proposed algorithm can effectively suppress the positive ratio(RP)of the "Deviation"phenomenon,while the average mismatching degree of the scene(η)is also reduced by at least 60%.