Pedestrian Detection And Tracking Technology Based On Lidar-Camera Fusion Model

Real-time and accurate pedestrian detection and tracking in the scenario of automatic driving is crucial for pedestrian safety.Under the mainstream trend of multi-source heterogeneous data fusion,cutting-edge models based on the data acquired by lidar sensor and vision sensor have achieved the effect of SOTA(state-of-the-Art)in terms of performance and indicators.We compare the cutting-edge models for pedestrian detection and tracking,include training,testing,analyzing the indicators and visualizing of false alarm samples and false negative samples.In the pedestrian detection research,we propose the anchor-based EPNet based on the multi-modal learning model EPNet and exceeded the current SOTA in indicators.In pedestrian tracking,Bi CNet-TKS model is proposed to optimize the pedestrian tracking algorithm 3D-MOT by trajectory matching.Our overall network for pedestrian detection and tracking developed by the combination algorithm of cutting-edge technologies.(1)According to the analysis of the two-stage model Point RCNN based on lidar point cloud data,we conclude that the model indicators are more susceptible to the influence of distance and occlusion than lighting conditions.(2)According to the analysis of the one-stage model YOLOX based on image data,we conclude that the model indicators can be influenced by illumination condition than partial occlusion and low resolution.(3)According to the analysis of multi-modal learning model EPNet,we conclude that multi-source information fusion can effectively improve the performance and indicators of the model.The Anchor-based EPNet based on metric learning algorithm introduce the dynamic anchor and advanced Consistency Enforcing Loss.(4)According to the analysis of the pedestrian Re-ID model BICNet-TKS,it is confirmed that the model still has strong robustness for automatic driving.In our algorithm,it is further used to optimize the target tracking pedestrian tracking algorithm.(5)Our integrated model for pedestrian detection and tracking improved by the embedding of cutting-edge tracker and track matcher,which achieves high performance,high indicators and stability.