Research On Subjective And Objective Quality Assessment Of Panoramic Video

With the development of software and hardware,virtual reality(VR)has popularized in the daily life as an important multimedia application.Panoramic video is one of the main media contents of VR.With the help of head-mounted display(HMD),it brings immersion by showing the 360°× 180°spherical video content to viewers,which is stitched with videos taken by multiple cameras at different angles.Due to the high resolution of human eye,extremely high resolution is required for the panoramic video in order to achieve realistic viewing experience.Higher resolution leads to massive data,posing challenges to the storage and transmission systems.Under this circumstance,quality assessment of panoramic video is crucial for finding the balance between limited resource and massive video data,and guaranteeing good quality of experience.For the processing of panoramic video,the methods for traditional video are still used,which require projecting the spherical video onto the plane according to a certain projection method before processing like compression,transmission.However,since the panoramic video is eventually viewed on the virtual sphere,the quality assessment methods for traditional plane video are not applicable for panoramic video.On the contrary,the subjective and objective quality assessment of panoramic video should be conducted in the spherical domain.In order to better reflect the visual quality in realistic viewing experience,this thesis conducts research on the subjective and objective quality assessment of panoramic video.Firstly,improved subjective quality assessment method is proposed considering the characteristics of panoramic video.Based on the test protocol of traditional video,an improved subjective quality method is designed by incorporating the characteristics of panoramic video viewing.More importantly,in subjective quality assessment,the appropriate presentation of the video sequences without introducing extra distortion is crucial to obtain reliable subjective quality scores.However,the fixed field of view of the HMD and the different video resolution lead to the sampling of the video sequences.On the other hand,the panoramic video is processed in plane domain while displayed in spherical domain.Since the sphere could not be transformed into the 2D plane without any deformation,the distortion brought by projection is inevitable during VR viewing.In order to alleviate the influence of sampling and projection on the subjective quality assessment,this thesis proposes the video resolution optimization method for subjective quality assessment by analyzing the geometric relationship between the virtual sphere and the plane screen.The optimal video resolution can help minimize the influence of projection and sampling,making the subjective quality assessment more reliable for panoramic video.Based on the improved subjective quality assessment method for panoramic video,a subjective quality assessment database termed IVQ(Immersive video quality database)is established and applied to the validation of full-reference objective VQA methods.Quality assessment can provide guidance for related technologies.Since the traditional VQA databases for plane video are not applicable to panoramic video,and few panoramic VQA database is available,this thesis first establishes a subjective quality assessment database based on the common test sequences released by Joint Video Exploration Team(JVET)and HEVC compression.The database is proved to be representative and reliable through the analysis on subjective data distribution and rating behavior consistency.The existing objective VQA methods for panoramic videos are compared and analyzed based on the IVQ database,results indicating that the quality assessment of panoramic video is still open for research.Finally,an objective quality assessment method is proposed for panoramic video.Differing from the traditional plane video,the panoramic video is viewed in the spherical domain.The visual quality of the projected plane cannot represent the real subjective quality in VR viewing.Therefore,panoramic VQA should be conducted in the spherical domain in order to better model the subjective quality.In this thesis,an NR VQA method is proposed for panoramic video.The proposed method extracts spatial and temporal video features from the spherical domain.Multiple kernel learning regression is used to map the video-level features into the objective quality score.The experimental results on the IVQ database verify that the proposed method can achieve higher correlation with the subjective quality compared with the NR method for 2D video,which proves the effectiveness of the proposed spherical-domain features on modeling the subjective quality perceived during VR viewing.However,the inferior performance to the FR methods indicates further improvement is still necessary.Future work may incorporate features that correlate better with the perceptual quality of panoramic video.