Perceptual Quality Assessment Of Omnidirectional Images

With the advent of 5G commercial era,Virtual Reality(VR)technology is constantly innovating and developing with vigorous momentum,and its influence is gradually spreading to all areas of life,activating industry applications and boosting market demand.As an important part of VR technology,omnidirectional images are also making their way into people’s daily life.Unlike traditional two-dimensional images which have flat and single scene content,omnidirectional images cover the whole 360°×180° viewing range,and users can freely explore the scenes with the help of head-mounted display for an immersive experience.The realism derived from the high resolution guarantees the quality of the immersive experience of the omnidirectional image.However,due to the practical factors,the quality of omnidirectional images is inevitably degraded in the process of shooting,transmission and storage.Therefore,it is important to measure the perceived quality of omnidirectional images for optimizing image processing sessions and improving the performance of VR devices.In this thesis,we conduct a comprehensive study on the visual quality of omnidirectional images from both subjective and objective perspectives.The specific research contents are shown as follows.(1)In order to fully investigate and analyze the impact of human viewing behavior and viewing conditions on the perceived quality of omnidirectional images,a large-scale subjective quality assessment database of omnidirectional images is established.The database contains258 original images captured with the Insta360 Pro2 camera,involving a total of 16 scene contents.After the initial processing,the database obtains a total of 1,032 distorted images by adding four types of distortion(Gaussian Noise,Gaussian Blur,Brightness Discontinuity and Stitching distortion)and three distortion levels.Based on this database,a large-scale subjective experiment is conducted in this thesis to collect users’ viewing behavior data and subjective ratings by setting different viewing conditions.Analyzing the subjective experimental data,we can find that the image quality is influenced by distortion types,viewing conditions and other factors.(2)Inspired by user viewing behavior and omnidirectional image characteristics,this thesis makes the first attempt to train a convolutional neural network based on the proposed database and proposes a no-reference omnidirectional image quality assessment model,which consists of three parts: viewport extraction module,multi-scale feature extraction module and perceptual quality prediction module.To be consistent with human viewing behavior in VR devices,viewport images are extracted from omnidirectional images as input data for the model,and incorporate the user’s viewing conditions naturally into the quality assessment process.The viewport images are subjected to local and global feature extraction in the multi-scale feature extraction module,and the features are subsequently regressed into quality scores by the perceptual quality prediction module.Finally,the overall quality of the omnidirectional image is obtained by averaging the predicted scores of all viewport images.The performance on the self-built database proves the effectiveness of the proposed model.