Environmental Perception Research Based On Millimeter-wave Radar And Camera Multi-source Information Fusion

With the progress of science and technology in today’s world,artificial intelligence is also developing rapidly.Since it was put forward in the 1970 s,autonomous driving technology has been an important field in the development of artificial intelligence.The technology of autonomous driving mainly includes three parts: perception,decision-making and execution.Perception is the premise and foundation of the whole autonomous driving technology,Only on the basis of realization of perception can we make corresponding path planning and driving behavior decision.Just as the name implies,perception is to feel and observe the road conditions surrounding the automated vehicle and recognize them.The common perception of sensors of automated vehicles are visual sensors(monocular and binocular cameras),millimeter wave radar,lidar and the perception system formed by different combinations of these sensors,etc.At present,most autonomous driving system in mass production use the environment perception algorithm based on single sensor.However,due to the advantages and disadvantages of each sensor,there are obvious limitations in using single sensor for environment perception.By using multi-sensor information redundancy and complementary advantages,the multi-sensor environment perception system can obviously solve the existing problems.Therefore,this paper studies the multi-source information fusion of millimeter wave radar and camera,and designs an environment perception algorithm based on the data fusion,including data analysis,target primary selection,target detection and tracking,sensor sampling cycle calibration and target data fusion.The main contents are as follows:(1)Using CAN protocol to analyze the original data of Millimeter wave radar and camera.In the target primary selection,this paper designs the target filtering algorithm,which can effectively filter out the empty target,noise signal and false alarm object in the original data of MMW radar.In the target detection,the commonly used algorithm is kalman filter.Kalman filter algorithm is suitable for linear,discrete and finite dimensional space,it can predict the coordinate position and velocity of objects from the observation sequences containing noise,and it is widely used in radar,computer vision and other engineering applications.However,the classical kalman filter algorithm has limitations on target detection and in practical application.This paper proposes a kalman filter with consistency check and life cycle decision-making,which can extract effective dangerous target from the original data and improve the accuracy of target detection and tracking.(2)Aiming at the problem of different signal sampling cycles of millimeter-wave radar and visual camera,we analyze the operating characteristics of these two sensors in practical applications,then set up kinematic equation and use an improved interpolation extrapolation method to calibrate and compensate the sensors’ perception vector,complete the time synchronization of heterogeneous sensors,which laid the foundation for further data fusion and improve the detection ability of perception system.(3)In multi-sensor signal fusion,the commonly used methods at present include weighting method,D-S inference,Bayesian estimation and track correlation algorithm,etc.In this paper,we start from the feasibility of heterogeneous sensor data fusion,adopt the decision-making level state vector fusion algorithm to fuse the signal data of millimeter wave radar and camera sensor,which make the perception system integrates the advantages of millimeter wave radar and camera sensor,enhance the performance of the whole environment perception algorithm.(4)In this paper,we build a joint simulation platform of Matlab/Simulink and dSPACE to simulate the road dataset obtained in China Automotive Engineering Research Institute Co.,Ltd.The simulation result shows that our proposed multi-source information fusion environment perception algorithm based on millimeter wave radar and camera sensor in this paper can effectively perceive the road environment outside the autonomous vehicle,and the real-time detection ability can be guaranteed also.At the same time,the visualization results after the real vehicle test indicates that the effectiveness and stability of the algorithm in the practical application of automated vehicle are verified.