Multi-task Detection Network And Multi-source Fusion Ranging For Complex Driving Scenes

Intelligent vehicles acquire spatial information about the driving environment by deploying vision sensors.However,in complex driving environments,vision sensors are susceptible to interference and data extraction techniques are not fully mature,leading to deviations in target detection,tracking and ranging,resulting in problems such as missed detection,false detection,poor accuracy and poor stability.This paper proposes a target fusion localization algorithm by building a multi-task detection network based on vision perception and using multiple sources of reference information,aiming to improve the perception capability of machine vision perception for complex driving environments.The research content of this paper is as follows:(1)Build a multi-task target detection network model based on vision-aware driving scenes.Considering the limited computational resources of the in-vehicle platform,this paper uses the lightweight network Mobilenetv3 to replace CSPDarknet53 as Back Bone and completes the design of lane line and vehicle detection loss functions,while 3D target detection is used to obtain depth information of images.(2)Proposed fusion algorithm for video stream data based on spatio-temporal continuous feature association.This paper investigates the matching and fusion based on the continuous feature association of video frame image information in the temporal and spatial domains: using Kalman filtering to predict the motion information of the target,fusing based on Io U,colour histogram,Marxian distance while using the Hungarian algorithm to achieve matching and tracking of the target detection frame.(3)A vehicle ranging algorithm based on multi-reference information fusion is proposed.The impact of camera position changes on the ranging accuracy of the vehicle detection frame based on the bottom edge position ranging model is analysed,the coordinates of the road vanishing point are calculated and the calculation models of the camera pitch angle and yaw angle are derived,based on which the ranging model is modified;the formula for calculating the lane width in the presence of pitch angle and yaw angle is derived,and the lane line based ranging model is modified under the bend;the license plate width based The 3D detection is used in the vehicle width-based ranging model to obtain the 3D information of the vehicle detection frame;finally,weights are assigned according to the confidence levels of the four ranging models in different application scenarios.(4)The accuracy and robustness of the multi-task target detection network model,spatiotemporal continuity fusion target tracking matching,and multi-reference information fusion ranging algorithm proposed in this paper are verified by real vehicle experiments based on the Xavier development board.The experiments show that: the average recall of the proposed multitask detection network slightly decreases,the average accuracy remains unchanged,the number of parameters is reduced by about 75%,and the average detection speed is improved by 15%;the accuracy of the tracking algorithm is 3.3% higher than that of the SORT tracking algorithm,the tracking frame accuracy is the same as that of the SORT tracking algorithm,and the number of transformations of the target vehicle tracking trajectory decreases significantly;the average computational speed of the fusion algorithm is The average computation speed of the fusion algorithm is 24 frames/second and the range measurement effect is more stable.The fusion ranging algorithm based on multi task detection network proposed in this paper improves the human like visual perception ability of intelligent vehicles,makes full use of the space-time continuity of targets and multi-source reference fusion ranging,greatly improves the speed of target detection and the accuracy and robustness of ranging with targets,and provides a guarantee for the safe driving of intelligent vehicles.