Research On Multi-Sensor Collaborative Perception Methods For Intelligent Driving

Road traffic safety issues,congestion issues,energy consumption issues,and pollution issues have always been important issues for social development and life safety.On the one hand,these issues have become more complex and serious with the increase of vehicles;on the other hand,emerging information technology and artificial intelligence have always played an important role in dealing with and improving the above issues,leading to the development of intelligent driving and intelligent vehicles faster and more important.From the perspective of intelligent driving,based on the new idea of multi-sensor collaborative perception of driving environment,this paper analyzes the technologies of road detection,object detection and object tracking in driving environment perception,and studies and proposes road detection,object detection and object tracking three aspects of multi-sensor collaborative sensing methods.The main work and achievements are as follows:(1)A road detection method based on the cooperation of monocular camera and lidar was researched and proposed.In order to improve the road detection effect,the lidar point cloud is first converted into an image by a method based on height difference,and then the converted image is fused with the camera image.Finally,a deep neural network is designed to segment the pixels of the fused image to obtain the road area.Experiments on the internationally recognized KITTI road dataset show that the proposed road detection method improves the detection effect.(2)A multi-sensor collaborative object detection method based on attention mechanism feature fusion was researched and proposed.In order to improve the effect of lidar-camera cooperative object detection,firstly,an image feature extraction neural network is designed to extract camera image features.Secondly,a fusion sub-neural network based on attention mechanism is designed to fuse the extracted image features with the features extracted from the lidar point cloud using the mature Point Net++network.Then,the region proposal network and refinement network of the Point RCNN network are adopted to obtain the final 3D predicted bounding box.Experiments on the internationally recognized KITTI 3D object detection dataset show that the proposed method outperforms other lidar-camera-based object detection methods.(3)A multi-sensor collaborative object detection method based on feature fusion and semantic augmentation was researched and proposed.Aiming at the problem that the point-based object detection method is time-consuming and the detection effect of difficult objects is not good,firstly,an image feature extraction process is designed to extract image features,and fuse them with the reflection intensity of the point cloud to obtain point-by-point fusion features.Second,the segmentation network segments the point cloud into foreground points and background point semantics according to the fusion features,and downsamples the point cloud through a semantically enhanced point set abstraction method.Then,the candidate points and their features are obtained through the candidate network.Finally,the predicted 3D bounding box and category are obtained by the object prediction process.Experiments on the KITTI 3D object detection dataset show that the proposed method improves the object detection effect on objects of medium difficulty level and difficulty level.In addition,by simplifying the image feature extraction process and removing the FP layer and refinement module of Point Net++,the detection speed is improved.(4)A multi-sensor multi-object tracking method based on linear programming was researched and proposed.In order to improve the effect of multi-object tracking,firstly,the proposed multi-sensor collaborative object detection method based on feature fusion and semantic enhancement is used to obtain the detection features of the object.Second,use the Kalman filter to predict the predicted features of the object in the next frame.Then,absolute subtraction is used to associate prediction features and detection features to obtain object relation features,and a similarity estimation process and startend evaluation process based on bidirectional long short-term memory(Bi LSTM)are designed.Finally,the optimal matching is obtained by means of linear programming.The experimental results on the internationally recognized KITTI object tracking dataset show that the proposed method is balanced in all aspects and has better comprehensive tracking effect.