Key Technologies Of Autonomous Driving Localization And Environment Perception System Based On Multi-information Fusion

Autonomous localization and mapping are the essential functions for autonomous driving to achieve high intelligence and the fundamental guarantee for the stable operation of autonomous driving vehicles.Not only the accurate and long-term localization in complex and changeable scenes is essential for control and real-time navigation,but the3 D map around the road can also directly help vehicles to understand environmental information and path planning.The 3D map is one of the core technologies of major autonomous driving manufacturers.Simultaneous localization and mapping(SLAM)technology is necessary to solve this problem and is widely used in robotics,virtual reality,and other fields.Although SLAM has made many remarkable achievements in academic circles in recent years,there are still many problems in practical application.Firstly,the slam system with a single sensor often has low robustness and can not adapt to the complex and changeable actual environment.Both camera and lidar data will degrade in specific scenes and then result in failure.Therefore,only by fusing a variety of heterogeneous sensor data can improve the system’s robustness to adapt to different scenarios.Moreover,a simple 3D map is not enough to meet the increasing intelligent needs of autonomous driving.Therefore,it is necessary to study the semantic mapping technology,fuse the perception data and enrich the representation information of the map.At this time,ensuring the geometric and semantic consistency of the map has become a complex problem to be solved.In addition,most work is generally based on public autonomous driving datasets.However,there is still much work to be solved in the existing system,such as sensor synchronization and calibration.This dissertation designs and implements a complete localization and environment perception system for autonomous driving to solve the above problems.The system integrates not only multi-source sensor data but also multi-dimensional feature information.In addition,this dissertation discusses the sensor fusion technology under different methods,especially the fusion of camera and lidar,and gives the solutions under neural network and geometric forms respectively.The main research contents of this dissertation can be summarized as follows:According to the actual requirements and technical indicators of the autonomous driving system,this dissertation proposes solutions for autonomous driving vehicle localization and environment perception system.The solution includes the selection of sensor hardware platforms,determining critical technical indicators,the calibration and processing of sensor data,and related software algorithms.The system can realize the accurate autonomous localization of the vehicle and express the semantic information in the 3D map.Unlike other localization and mapping systems based on a single sensor,this dissertation uses three sensors: lidar,camera and inertial device for fusion positioning,which overcomes the disadvantage of low robustness of a single sensor but also brings more complex data processing difficulties.Aiming at the difficulty of extrinsic calibration of lidar and camera,this dissertation proposes an end-to-end lidar-camera online extrinsic calibration network,which can automatically complete the extrinsic calibration of lidar and camera without relying on artificial markers.Aiming at the weak generalization ability of the existing pose estimation network,this method combines the traditional pose estimation method based on geometric constraints with the neural network.It embeds the nonlinear optimizer into the end-to-end network for training.To ensure that the neural network can give full play to its feature extraction ability,this method uses the corresponding feature extraction network for 2D data and 3D data respectively,and realizes the data association from 3D to 2D through keypoint selecting and feature matching network.In this method,the traditional neural network is only responsible for feature extraction and association,which can deal with changes in the environment and sensor parameters.Finally,experiments show that this method has stronger robustness and generalization ability.Aiming at the low accuracy of the existing lightweight 3D semantic segmentation network,a real-time 3D semantic segmentation network integrating lidar and camera data is proposed in this dissertation.Unlike the processing methods in the previous part,this method adopts the front fusion scheme,makes a unified representation of lidar and image data through spherical projection,and learns the fusion methods of different feature channel data through the neural network.To improve the semantic segmentation accuracy of lightweight networks,this method also introduces a spatial module to make up for the lack of spatial information caused by channel clipping.Finally,a series of experiments are designed,which proves that the network greatly improves the accuracy compared with the original method and deeply discusses the neural network’s ability to process different channel data.Aiming at the problem that a single sensor’s localization and mapping method can not adapt to the complex and changeable actual environment,this dissertation designs and implements a localization and mapping method based on multi-information fusion.This method integrates the data of the camera,lidar and inertial devices,and fully considers the different geometric primitive features in the environment and greatly improves the accuracy of feature correlation by using the various constraints generated by 3D points,lines and surfaces.The constraint form and the Jacobian matrix of different sensors are carried out in this dissertation.A global factor graph back-end is constructed to interpret different sensor constraints.In addition,this method also designs loop detection and re-localization methods along with sensor degradation detection and processing methods to improve the system’s robustness further.To integrate the high-dimensional semantic features,this dissertation constructs a semantic weight factor,which combines the semantic constraints and geometric constraints to ensure the semantic and geometric consistency of the map.Finally,this dissertation makes sufficient experimental verification and detailed analysis for the autonomous driving localization and environment perception system mentioned above and compares it with the other two influential localization and mapping methods.Through the test of the public dataset,it is proved that the system has significant advantages in accuracy compared with the method of a single sensor.Through a series of tests of actual autonomous driving vehicles in the campus environment,it is proved that the system has better performance in accuracy and robustness,and can still work stably even in the sensor degradation scene of the underground parking lot.