Front Vehicle Detection Based On Millimeter Wave Radar And Vision Fusion

Forward vehicle detection has been an important part of research as a key point of environmental sensing technology in vehicle assisted driving.In the Forward vehicle detection,millimeter wave radar and camera sensors are widely used.Millimeter wave radar has the advantage of being weather-proof and it can work around the clock and has a long detection distance.However,it is difficult to identify the target.The camera can recognize the target quickly,but its detection distance is limited.Besides,the environment is easy to influence the camera's working efficiency.In view of the shortcomings of target detection for the above single sensor,this dissertation proposes a forward vehicle detection method based on millimeter wave radar and visual fusion.Compared with traditional region generation vehicle detection method,the method proposed in this dissertation has low computational complexity and short detection time,which can meet the requirements of real-time.The main research contents are as follows:(1)Primary-selection of forward vehicle targets based on millimeter wave radar.Firstly,the detection principle of millimeter-wave radar for dynamic and static targets is studied.Through selection of the model and performance parameter analysis,we select the CAR150 millimeter wave radar of NANORADAR as the radar used in this dissertation.Then I analyzed the process of millimeter wave radar data analysis and the deletion of radar jamming targets.Finally,the effective vehicle target is selected based on the distance and lane information.(2)Forward vehicle detection based on convolutional neural networks.Firstly,I studied the basic principles of R-CNN and Faster R-CNN algorithms.The forward vehicle detection method based on the improved Faster R-CNN algorithm is proposed innovatively.It uses two RPNs,which can take the region of interest generated by the first RPN as the new anchors of the second RPN,and extract more accurate region of interest(ROI),which can achieve accurate classification and location.The real vehicle collects vehicle images from different scenes and labels with LabelImg tools to produce standard vehicles.The data set is finally verified by experiments.The results show that the proposed algorithm has a higher average accuracy(AP)than the original algorithm,reaching 96.3%,and the detection time of each image is 0.05s,which satisfies the requirement of real-time detection.(3)The forward vehicle detection model based on millimeter wave radar and visual fiusion is built.Firstly,I deduce the transformation between millimeter wave radar coordinate system and image pixel coordinate system.The internal and external parameters of the camera are calibrated by Zhang's calibration method,and the joint calibration of millimeter wave radar and camera is completed.The neural network is innovatively proposed.The joint calibration method of millimeter wave radar and camera based on neural network is innovatively proposed,which can reduce the complexity of calibration.Based on the data collected by millimeter-wave radar with low sampling frequency,the time synchronization between millimeter-wave radar and camera data is realized by multi-threading.According to the aspect ratio of the vehicle,the radar projection ROI is determined.The ROI is further filtered by the vehicle ROI determination algorithm to realize the vehicle detection of millimeter wave radar and visual fusion.The average detection rate of the algorithm under sunny,cloudy and night weather conditions respectively 96.1%,95.6%,94.8%,the vehicle detection accuracy is high,and the average detection time of single-frame image data is 30.6ms,31.3ms,31.7ms,respectively,which satisfies the requirements of real-time.