Research On 3d Object Classification Algorithm Based On Panoramic Image

As the core algorithm in three-dimensional understanding,the three-dimensional object classification algorithm is also widely used in actual production and life,such as human-computer interaction technology and autonomous driving technology.This thesis takes the task of 3D object classification as the goal,and focuses on the 3D object classification algorithm based on panoramic images.This thesis explores the local information of the panorama,the multi-modal fusion of the panorama and point cloud,the rotation invariance of the panorama and proposes an improved deep learning algorithm.The key contributions of this thesis are as follows:1.Explore the local information of the panorama.On the basis of the original algorithm of spatial distribution panorama of point cloud,this thesis introduces the generation algorithm of spatial gradient distribution map of point cloud,which combines the size information of three-dimensional objects.Then,the single-channel spatial distribution panorama,vertical direction gradient map and horizontal direction gradient map are pieced together into a three-channel panorama in order.For each three-dimensional object,we projected three three-channel panoramic images based on the X,Y and Z axes in the spatial Cartesian coordinate system.At the same time,we take the VGG16 two-dimensional image classification network as the backbone network and propose a multi-branch deep learning network to fully learn the high-order features of each panorama.Finally,the multi-channel panorama corresponding to the 3D object is fed into the multi-branch deep learning network to obtain the classification result of the 3D object2.Explore the multimodal fusion of panoramic image and point cloud.Based on the original 3D object classification algorithm based on panoramic image,this thesis introduces the 3D object classification algorithm Pointnet based on point cloud,and makes the decision-making fusion of 3D object panoramic image mode and point cloud mode.For each 3D object,we first obtained the three-channel panoramic image based on X axis,Y axis and Z axis projection according to the multi-channel panoramic im-age generation algorithm described in the first work.Then,the point cloud data is up-sampled to ensure that the number of point clouds sent into the deep learning network is uniform.Finally,the multi-channel panorama corresponding to the 3D object and the normalized point cloud are sent into the multi-mode deep learning network to obtain the classification result of the 3D object.3.Explore the rotation invariance of the panorama.In this thesis,based on the original multi-channel panorama generation algorithm,a panorama calibration module based on Gaussian curvature value is introduced.The main function of this module is to calibrate the input 3D object to ensure that no matter how the 3D object is rotated and translated,the calibration is the same orientation Angle.In this way,the final projected panorama of the three-dimensional object has rotation invariance.