Research And System Design Of Millimeter Wave Radar And Camera Fusion Vehicle Detection Based On Feature Weighted And Visual Enhancement

Intelligent transportation system is one of the important means to solve traffic congestion,traffic pollution and traffic accidents.Roadside perception is used to identify vehicle targets on the road and provides road information for the intelligent transportation system.It is an important part of the intelligent transportation system.In roadside perception,millimeter-wave radar is often combined with cameras to obtain richer road information.On the one hand,the roadside perception system requires vehicle target detection throughout the day,especially in low light,to maintain high detection accuracy;on the other hand,it requires long detection distance coverage to reduce the number of detection equipments deployed.Aiming at these problems,this paper studies a millimeter wave radar and camera fusion vehicle detection method based on feature weighting and visual enhancement,and designs a vehicle target detection system for roadside perception.The main research contents of this paper are as follows:(1)Aiming at the problem of low detection accuracy caused by the small proportion of long-distance vehicles in the image,the lack of visual information and the difficulty of extracting effective features,this paper proposes a visual enhancement method based on millimeter-wave radar spatial preprocessing.The method starts with the fusion of millimeter wave radar and camera data layer,characterizes the spatial locations of potential targets based on millimeter-wave radar,and uses the characterization results for the division of long-distance target areas in visual images.Further,the image of the divided area is reconstructed,detected and restored to improve the visual detection accuracy of long-distance targets.(2)Aiming at the problem of the lack of target detail features in the image in low light environment,which leads to low target detection and recognition rate,this paper proposes a feature-weighted millimeter wave radar and camera fusion method.Considering the difference in the contribution of different layers to feature detection,the weight parameters of different feature maps are obtained through model training,and the features of different layers are fused and calculated according to the weight to enhance the feature information of the target.Next,add branch networks and use convolutional layers of different sizes to extract different receptive field information in the feature map,and the branch output results are fused to obtain stronger image representation ability,which can achieve the goal of improving the detection accuracy in low light.Then,combined with the featureweighted millimeter wave radar and camera framework and the vision enhancement based on millimeter wave radar spatial preprocessing,a millimeter wave radar and camera fusion detection network based on YOLOv4-tiny is constructed.Finally,distance information is added to the vehicle target through information fusion.(3)In order to verify the feasibility of the millimeter wave radar and camera fusion vehicle target detection algorithm proposed in this paper,a set of embedded vehicle detection system for roadside perception is designed.First,the millimeter-wave radar target is mapped to the image through sensor time and space calibration.On this basis,the detection of the vehicle target by millimeter wave radar and camera fusion is realized,and the position of the vehicle target on the image is obtained.The experimental results show that the average precision(AP)of the proposed algorithm is improved by 5% compared with the millimeter wave radar and camera fusion algorithm RVNet when detecting vehicle targets within the distance of 80 meters in low-light environment.In the detection performance tests at different distances,the AP value of the algorithm in this paper is 63%higher than that of RVNet when detecting 120-meter targets.