Research On Environment Perception And Prediction Of Intelligent Driving System Based On In-Vehicle Platforms With Different Computing Power

As the inputs to the intelligent driving system,the environment perception and prediction are indispensable and critical components.Perception is the detection of elements in the current environment,and prediction is the anticipation of these elements in the future period,which is also part of the environment perception.Different levels of intelligent driving systems have variability in the number of perception tasks and algorithm performance requirements,and are also constrained by the computing power of in-vehicle computing platforms.Currently,the general trend of in-vehicle computing platforms is to use large computing power to support the deployment of large-scale deep learning models in order to achieve higher algorithm performance and more complex and stable functions.But at present,high-end chips are only installed in a small number of flagship smart vehicles,and there is still the necessity and a large market demand for chips with high energy efficiency.Therefore,within a period of time in the future,the intelligent driving environment perception algorithm and computing platform will still maintain the coexistence of high and low computing power computing platform to meet different requirements.Currently,the perception and prediction algorithms in smart driving systems are still limited by both performance and real-time.In terms of scientific research,there is still room for improvement in algorithm and model performance.In terms of engineering applications,the real-time performance and lightweight of algorithms and models also need to be further improved.In this paper,we take the perception and prediction in intelligent driving system as the research target,considering low and high computing power in-vehicle platforms,and carry out research on environment perception and prediction methods based on the premise of real-time and lightweight of algorithms and models,including machine learning and deep learning methods.For the low computing power platform,the algorithms related to target detection,tracking,fusion,and lane line detection tracking were innovated using automotive millimeter wave radar and vision as sensor inputs,and implemented and tested on Toshiba’s in-vehicle computing platform.For the high computing platform with vision as sensor input,first,a lightweight multi-task learning model architecture is proposed and a design paradigm for multi-task network architecture is constructed.Then,a task loss optimization method for model training is proposed based on machine learning algorithms.Finally,an attention module and multimodal trajectory and driving intention prediction model for vehicle interaction are proposed.The above research on deep learning balances model complexity,memory throughput,and inference efficiency.Details of the research include.(1)A 3D automotive millimeter wave radar and vision fusion perception is constructed.First,a 3D automotive millimeter wave radar simulation model with elevation measurement capability is built based on System Vue,and the effectiveness of the model and target detection algorithm is confirmed through simulation experiments.The radar prototype fabrication and algorithm development were completed based on ST’s chip solution,and the average error of target height detection was ±5%.Then,the target detection algorithm was developed based on Toshiba’s in-vehicle computing platform,and the adaptive hash-based tracking algorithm and the monocular ranging algorithm with fused vanishing point online calibration were proposed.A multi-step detection system and a target life cycle model were constructed.A training sample collection,labeling and optimization tool chain is developed to optimize the dataset by sample cleaning and difficult sample mining,and the vehicle and pedestrian detection recall rates are 95.32%和 90.29%,respectively.DBSCAN is improved for millimeterwave radar detection point clustering to improve the detection of low-speed moving targets.Finally,a target-level fusion method for camera and millimeter-wave radar is given.Through experiments in real scenarios,the results show that the fusion method improves the target detection accuracy with the average errors of 0.25 m and 1 m for lateral and longitudinal positions,respectively;the maximum percentage average error in longitudinal direction is 2.9%;and the average frame rate of the system operation is 72ms@13.9FPS.(2)Based on the constructed spatio-temporal domain and particle filter,a lane detection and tracking system(Spatial-Temporal Particle Filter based Lane Detection and Tracking System,STPF-LDTS)is proposed,which is implemented on the same Toshiba in-vehicle computing platform.A more general multi-lane weak model is proposed to improve the flexibility and robustness of the algorithm.The spatiotemporal domain representation by reconstructing image sequence makes full use of the historical spatial information to integrate the detection and tracking process of lane markers,which reduces the interference noise of the environment.A weak prior knowledge-based lane marker candidate detector and a particle filtering-based tracking algorithm are proposed.In particular,the Euclidean distance transformation algorithm for confidence map generation and update is improved for efficiency,and it is used for particle sampling and related weight calculation.Experimental results on open datasets show that the proposed algorithm achieves optimal performance compared to machine learning algorithms,with an average accuracy of 98.91%.The lightweight implementation of the algorithm was performed on the Toshiba in-vehicle computing platform and tested online on real roads.The results show that the algorithm works stably and runs at an average frame rate of 60ms@16.7FPS.(3)For multitask learning,a lightweight Uniform Multi-Task Network(UMT-Net)and an Adaptive Task Weighting(ATW)optimization strategy are proposed,and the impact of different feature routing on model performance is analyzed.Self-Attention(SA)module,Joint-Attention(JA)fusion module,and task-specific feature aggregation decoder are proposed in the model architecture.For the optimization problem of multitask loss,a model loss function is derived based on the maximum a posterior(MAP)probability of the model parameters inspired by machine learning methods,in which the prior distribution of the model parameters is approximated.Experimental results on the City Scapes and NYUv2 datasets show that the proposed model outperforms all compared baseline networks,achieving the state-of-the-art(SOTA)performance with less computation,model parameters,and inference time.The mini-version of the model uses about 2.4% of the computation and 1.9% of the model parameters of Dense Net to achieve approximate performance with a theoretical inference speed of 300+FPS.In addition,the proposed task loss optimization method can improve the performance of all models by more than other weighted methods and can provide a more stable training process during the training process.(4)A high-performance visual multi-task network perception model(Uniform Multi-Task Network for Autonomous,UMT-AUTO)for autonomous driving scenarios is proposed,in which the perception tasks contain: 13 types of target detection and recognition,drivable area segmentation,lane line,crosswalk,stop line and road edge segmentation,and scene depth estimation.Based on the proposed multi-task network architecture and many high-performance modules,global shared backbone networks,task branching networks,fusion networks,and decoders for different tasks are designed,respectively.Combined with the ATW optimization method for training,the model results surpass currently the best multi-task network YOLOP for autonomous driving scenarios.specifically,on the BDD100 K dataset,the recall for the target detection is improved by 3.2%,the m AP50 is improved by 3.1%,the m IoU for the drivable area is improved by 2.6%,the accuracy for the lane line related task is improved by 6.6%,IoU improved by 2.3%,and the inference speed was 25.8ms@38.7FPS.In addition,the experimental results of generalizability in various practical scenarios around Jiading District,Shanghai,show that the model has good generalization performance and can still output satisfactory inference results in completely unfamiliar environments.(5)A Multi-Modal Trajectory Prediction Network(MMTP-Net)based on attention mechanism is proposed for multimodal trajectory and driving intention prediction.Indepth comparative analysis of the effects of different interaction tensor construction methods and the different model inputs on the network performance are presented.A lightweight Interaction Attention Module(IAM)for vehicle trajectory prediction is proposed.Experimental results on the US101 dataset with the ATW optimization method show that the proposed model achieves the best performance among all CNNbased methods.Compared with the baselines,the model predicts the trajectory of the vehicle in the next 5 seconds with a reduction of 0.4 m in the RMSE metric for the latreral and longitudinal errors;the accuracy of the lateral and longitudinal driving intention prediction is improved by 1.05% and 2.18% on average,respectively;meanwhile,the model computation and number of parameters are reduced by 20.32%and 24.18%,respectively.The average inference time is 6ms@167FPS.The results show that the intelligent driving perception and prediction methods in the paper for low and high computing power platforms can better meet the requirements of perception tasks under different computing power platforms,provide higher algorithm performance while ensuring efficiency,and have a promising applications prospect.