Research On Object In Front Of The Vehicle Detection Methods With Vision And Mm Wave Radar Fusion

With the frequent occurrence of automobile traffic accidents,in recent years,automotive active safety technology with advanced driver assistance and automatic driving as the core has received extensive attention and research around the world,among which accurate and real-time vehicle front objects detection has become one of the key technologies of research.This paper studies the detection method of the objects in front of the vehicle based on the fusion of vision and millimeter-wave radar,and successfully achieved a comprehensive and intuitive detection of the objects in front of the vehicle,this effectively solved the problem of high missed detection rate when a single sensor detects different types of objects in different weather environments.The main contents of this paper are as follows:（1）Visual detection of objects in front of the vehicle.Under the Pytorch deep learning framework,using the YOLOv4-tiny object detection algorithm to perform multiple sets of training on the expanded VOC format dataset combined with multiple training techniques,then use the weight file with the highest m AP value in the training result for visual detection,the localization and classification of three categories of objects such as person,car and bus in front of the vehicle are well completed.（2）Acquisition of effective objects for millimeter-wave radar.According to the functional characteristics of the selected model of millimeter-wave radar,the message information returned by the radar is analyzed with the specified protocol,then according to the analyzed radar data removed the empty objects,jamming objects and non-hazardous objects,and the relative distance and relative speed information of the effective objects in front of the vehicle are obtained.（3）Space-time fusion of camera and millimeter-wave radar.Firstly,Adopting the decisionmaking fusion level and the parallel fusion structure as sensor fusion scheme;secondly,according to the positional relationship between the camera and the radar installed on the vehicle and the imaging principle of the camera,the transformation relationship between the coordinate systems of the two sensors is determined and the radar scanning point to the mapping of the pixel coordinate system is realized;then using the visual detection frame match the projection point of the radar scanning point in the pixel coordinate system to complete the association of the two sensors to the same object;finally,the appropriate sampling point is selected according to the sampling frequency of the two sensors to complete the time fusion.（4）Built an experimental platform and experimental verification.Built a vehicle experimental platform equipped with camera and millimeter-wave radar,then carried multiple experiments in different weather environments and compared with single sensor,which proved the effectiveness of the fusion method in this paper for target detection in front of the vehicle.