Detection And Segmentation Of Panoramic Image Based On Spherical Convolution

With the rise of VR(Virtual Reality),panoramic image which is the content carrier of VR has become the focus of researchers.A panoramic image can be regarded as an image wrapped on a sphere,which needs to be projected onto a plane for storage and viewing.Equirectangular projection is the most important projection method for panoramic images.It maps the longitude and latitude lines of spherical images into equally spaced vertical lines and horizontal lines,thus causing tensile deformation of images,especially in the polar regions.Since the traditional convolutional neural network is designed for traditional planar images,it lacks generalization ability for image deformation.Therefore,this panoramic image brings challenges to two important tasks in computer vision:object detection and semantic segmentation.In this thesis,a spherical convolutional neural network is investigated and improved to solve the distortion problem of panoramic images.This spherical convolution changes the sampling position of the convolution kernel by mimicking the projection process of the panoramic image based on the projection relationship of the sphere and the plane.The spherical convolution makes full use of the geometric information of its deformation and can extract features better than traditional convolutional neural networks.Since it only changes the way the convolutional kernel is sampled,spherical convolution is used in the same way as traditional convolution.To verify the rationality of this convolution design,this thesis firstly uses it for the classification of panoramic images,and through the comparative experiments we show that spherical convolution can better classify panoramic images compared to traditional convolutions.Then the spherical convolution is applied to the classical feature extraction network framework to the object detection and semantic segmentation of panoramic image,training and detection on the panoramic image dataset.Experiments show that the spherical convolution can be well applied to the existing network framework,and can improve the accuracy of detection and segmentation of panoramic image.