Vehicle And Pedestrian Detection And Tracking System Based On Multi-sensor Fusio

In the driving process of autonomous vehicles,whether vehicles and pedestrians can be detected and tracked quickly and effectively is one of the important issues to be solved by vehicle environment perception technology,and it will directly affect the safety of autonomous vehicles in practical applications..Accordingly,based on the analysis of domestic and foreign vehicle and human detection and multi-target tracking algorithms,this paper proposes a multisensor fusion system suitable for automatic driving,capable of detecting,ranging and multitarget tracking of vehicles and pedestrians..The main research contents are as follows:(1)Aiming at the ranging problem in the field of autonomous driving,a multi-sensor data fusion method is proposed.Through the SGBM binocular matching algorithm,the ranging range of the binocular camera is improved.Through the joint calibration of lidar and binocular camera,The point cloud data is projected onto the image,and the binocular ranging results are corrected to obtain distance information with high frame rate and high accuracy,which solves the problem of single sensor ranging defects.(2)Improved the YoloV4 algorithm.By introducing a collaborative attention mechanism,the model’s ability to acquire features of each image channel is improved,and position information is embedded in the channel attention,which can not only capture cross-channel information,but also capture directional perception and Location-aware information enables the model to show better performance when detecting small targets and occluded targets.Collect and establish vehicle and pedestrian detection data sets,learn the front,side,and back of the vehicle as different categories,and directly determine the driving direction of the vehicle through the detection results.(3)Improve the Deep SORT algorithm.By introducing the distance information of the target,the two-dimensional target tracking is converted into the three-dimensional target tracking.The distance judgment mechanism is added in the matching cascade process,which effectively solves the problem of dense target tracking due to the close location.The number exchange that appears.(4)Build a vehicle and pedestrian detection and tracking experiment platform to verify the detection and tracking algorithm proposed in this paper.The experimental results show that the algorithm in this paper achieves an average real-time detection and tracking frequency of 33 Hz on the NVIDIA GEFORCE RTX 2060 computing platform.It has good detection and tracking results in these environments.The vehicle pedestrian detection and tracking system based on multi-sensor fusion researched in this paper has practical application significance for autonomous driving and related autonomous mobile platforms.