Research And Implementation Of Deep Learning 3D Target Detection Method Based On Point Cloud And Image Features

3D object detection is a key technology in the field of autonomous driving,which can detect the position and shape of targets in 3D space.The use of pure point cloud or image data for 3D object detection has its limitations.Many researchers have therefore tried to use multimodal data to improve the detection results.Based on the theory of deep learning,this paper studies the 3D object detection method using lidar point clouds and images.The main research work and innovation of this article are as follows:1)Analyzed the key technologies and main processes in 3D target detection,researched on detection methods using multi-modal data,designed a detection framework,and established Retina Net and Point Net ++ networks as research directions through further analysis.2)An improved Retina Net two-dimensional target detection network based on the Attention mechanism is proposed.This network strengthens the target detection effect of the Retina Net network by adding trainable filters to the background CNN model output to suppress background features and highlight target features.Using the KITTI dataset to test on various back-end CNN models,Attention-Retina Net has shown performance improvements.3)An improved Point Net ++ point cloud feature extraction network based on FPN is proposed.The network borrows the idea of constructing a feature pyramid from FPN,and builds a top-down branch network next to the Point Net ++ backbone network to perform multi-scale feature extraction on point cloud data.The feature extraction capability of the network is verified by object classification in the Model Net40 dataset.The experiments show that the multi-scale features output by the FPN-Point Net ++ network have stronger feature expression capabilities and achieve higher accuracy in point cloud object classification.4)A three-dimensional target detection method based on point cloud and image feature fusion is proposed.This method uses a novel projection method to project image features onto the point cloud.The first two research results of this paper are connected in series to achieve three-dimensional target detection..First,Attention-Retina Net is used to perform object detection in the image to obtain the two-dimensional bounding box of the target.Then,for each two-dimensional bounding box,the image features are projected onto the point cloud of the corresponding area in the three-dimensional space according to its viewing angle.The FPN-Point Net ++ proposed in this paper extracts features from the point cloud in the area and returns to the 3D bounding box of the target.