Quality Evaluation And Enhancement Of Immersive Visual Media Experience

Highly immersive visual media brings users an unprecedented immersive visual and psychological experience,and therefore its experience quality evaluation is more complicated than traditional flat video.Traditional flat video and binocular stereo video only involve visual single-channel perception.When a user views stereoscopic video,binocular visual perception characteristics such as binocular fusion,binocular competition,and binocular suppression are stimulated.However,when the user interacts with an immersive virtual environment,the change of the head movement speed will stimulate the vestibule system,and the body movement will stimulate the muscles and other motor organs,which will work together with the visual channel stimulation to stimulate a series of multi-channel perceptions.Therefore,the experience quality evaluation of immersive visual media is a multi-dimensional problem including the quality of visual single-channel media content,visual-vestibular-proprioception and other multi-channel experience quality and interaction quality.Faced with this complex psychological science and information science problem,it is necessary to divide and conquer the multi-dimensional quality indicators,explore the perception processing mechanism and perception characteristics of each dimension and establish a quantitative calculation model,and then gradually expand and complete the theory of immersive visual media experience quality analysis.At the same time,the huge data volume and ultra-low latency requirements pose great challenges to the efficient encoding and transmission of immersive visual media systems.In order to provide theoretical and technical supports of digital new media applications for the further development of immersive media applications,quality evaluation and enhancement of immersive visual media experience are essential research topics.This paper aims at the frontier issues about immersive visual media content application,focusing on monocular vision single-channel,binocular vision single-channel,visual-vestibular multi-channel perception characteristics and quality evaluation model,and subjective experience optimization.The following innovative results have been achieved:1)The first image subjective quality evaluation database including the latest hybrid coding and intelligent coding distortions is establised.In-depth studies reveal the statistical difference and human perception difference between reconstructions of learned compression and block-based compression.An objective evaluation neural network model for the perceptual quality of learned-based compressed images is proposed,which uses the image codec classification task to enhance the model’s ability to represent the coding distortion of neural networks,and predicts the quality of visual single-channel images by comprehensively combining deep features and manual features.The model achieves leading performance than existing quality assessment algorithms on the built database.The consistency between the model predicted quality score and the subjective score reaches 0.92.2)A binocular stereoscopic image depth perception evaluation algorithm is proposed,which simulates the formation process of depth perception in the human visual system according to the processing mechanism of visual physiology and psychology,and considers the characteristics of binocular fusion perception.The depth features that can reflect the depth information perceived by the human visual system are extracted from monocular cues and binocular cues.The evaluation consistency between the algorithm and subjective perception reaches 0.82,which exceeds the performance of the existing benchmark algorithms.It can be used to guide the binocular stereo content generation and content processing,and optimize the binocular visual channel experience jointly with the visual comfort evaluation index.3)A new visual-vestibule multi-channel perception subjective measurement method is designed,which uses the active speed guidance method combined with audio-visual stimuli to effectively guide the subjects to move smoothly with the target speed,and therefore to measure the perception under specific motion speeds.With the aid of this subjective measurement method,the previous qualitative analysis on the influence of actual motion speed on spatial synchronization perception was broken,and a computational model of spatial synchronization perception influenced by actual head rotation speed was measured and established through psychophysical experiments.4)A viewport spatial orientation adaptive technology based on spatial synchronization perception for 360-degree video adaptive streaming systems is proposed to reduce the sensitivity of the system to user behavior prediction errors.This technology is pioneering from the visual-vestibular-proprioceptive multi-channel perception characteristics of humans in the virtual environment.It fully exploits the multi-channel perception redundancy by using the spatial synchronization perception threshold model,adopts the virtual rotation velocity mismatch strategy in video rendering to introduce an imperceptible pseudo-motion response to control the displayed viewport switching angle,reducing the demand for high-quality transmission areas of panoramic video,and therefore significantly improving the viewing window quality under limited bandwidth conditions.The proposed method achieves a nearly 1.43% improvement in encoding and transmission performance.