Real-time Perception Technology For Urban Autonomous Vehicles Based On Lidar And Vision Fusion

Self-driving vehicles are stripped of the most unstable human factors in the driving environment and are considered to be the most important future development direction of the automotive field.The main technologies can be divided into three parts: perception,decision and control.As a basic work in the key technology of autonomous driving vehicles,environmental perception is of great significance for improving the environmental understanding of autonomous vehicles and ensuring their driving safety.At this stage,perception algorithm relies heavily on powerful device computing power,and rarely considers the economic requirements of autonomous driving systems,making it difficult to deploy to embedded platforms.At the same time,pedestrians and vehicles are mixed in real urban scenes.Existing single sensor sensing algorithms have the problems of insufficient sensing dimensions,poor adaptability of the algorithm environment,and weak robustness.Therefore,reducing the demand for the computing power of the sensing technology and improving the accuracy and robustness of the sensing algorithm in real scenes based on multi-sensor fusion are the keys to achieving autonomous driving technology in urban scenarios.Based on the Zhishan Intelligent Vehicle Test Platform of Southeast University,this paper designs a real-time sensing technology for lightweight autonomous driving platform,and studies a high-search full-rate real-time sensing algorithm based on the fusion of lidar and vision to realize the perception of real vehicle applications in urban street scenes.The specific research content is as follows:(1)Research on the real-time target recognition of YOLOv3 pruning algorithm for embedded devices.Aiming at the difficulty that YOLOv3 network is difficult to deploy on embedded platforms,the characteristics of YOLOv3 backbone network have been studied in depth.The network pruning method based on BN layer coefficients is introduced to add the L1 regularization term to the loss function and punish the BN layer gamma coefficient.The importance of the product channel is judged,unimportant connections and convolution channels are clipped,reducing the redundancy of the YOLOv3 network and completing the network slimming.(2)Research on real-time target detection of 3D lidar based on variable parameters.Through the acquisition of the lidar raw data,the appropriate filtering parameters are selected for downsampling,and the slope threshold segmentation algorithm is used to complete the ground segmentation.Aiming at the problem of cluster under-clustering and over-clustering caused by fixed parameter,a model of clustering radius and distance is proposed to improve the accuracy of clustering.Aiming at the problem of target 3D information extraction after clustering,the idea of image envelope is innovatively introduced to extract the minimum envelope rectangle of the 3D point cloud to achieve accurate target 3D information extraction.(3)Research on high-recall real-time target detection based on multi-sensor fusion.The lidar and camera’s own coordinate system is established,and the conversion relationship between the coordinate systems is obtained based on the joint calibration.The multi-thread and time-stamp synchronization technology is used to complete the time matching.According to the characteristics of high recall of lidar,a method for verifying visual targets using radar is proposed to improve the accuracy of visual recognition in poor light environments.Finally,the enhanced visual target information and three-dimensional target information fusion strategy is designed to achieve sensor information fusion.The actual vehicle test results show that the target detection algorithm after pruning greatly improves the real-time performance without reducing the accuracy,and meets the deployment requirements of the embedded platform.The radar algorithm can complete the3 D target detection and pose estimation.The multi-sensor fusion algorithm has higher accuracy than a single sensor,and the fusion recognition system can be deployed on a real vehicle to meet the urban street scene perception requirements.