Research On Omnidirectional Video Processing And Presentation Architecture And Key Technologies

With the development of multimedia and network transmission technologies,the applications of video are more and more popular.Due to the abundant visual expression,the video has played a leading role in Internet communication.However,traditional videos are gradually unable to meet people's new visual needs.In recent years,virtual reality technology is booming.The integration between interactive virtual omnidirectional video scenes and the actual behavior of the user greatly enriches the user's experience.Meanwhile,the continuous innovation of terminal display devices provides the possibility for the presentation of virtual reality content.Although the demand for omnidirectional video applications is increasing,the processing and rendering services of omnidirectional video face many challenges.First,omnidirectional videos with 8K or even higher resolution demand a high consumption of transmission bandwidth.For example,the required bitrate of an encoded video stream with a resolution of 2K is generally about four Mbit/s.If the resolution reaches 4K,the bitrate will increase to 20 Mbit/s.Meanwhile,the range of perspectives is limited,which is impossible to view the content of all directions at one time.Users only pay attention to a small proportion in the whole area.Therefore,a cloud-based view-adaptive video processing transmission scheme is proposed in the paper.According to the viewing information,the higher bitrate will be allocated in the field of view,and the background area is in a low quality.The solution can effectively reduce bandwidth requirements,alleviate the black field problem,and is compatible with traditional client players.Under this scheme,the video frame will be spatially divided into multiple tiles.In order to avoid the fragmentation of video content on the server side,we encapsulate multiple video tracks in the container layer,and propose an omnidirectional container-analyzing scheme,which can extract tiles from high and low quality video version,and recombine multiple tile streams in real time according to the view information.Finally,we connect the server-side transmission system with the client-headed device to form a closed loop,which realizes the interaction between the perspective information and the video content.This paper specifically introduces the design details and optimization of this view-adaptive architecture.Furthermore,we made some experiments to evaluate the performance of the system's bandwidth and quality.There usually exist delays in the network transmission and system modules.The server generates the mixed-rate video segment according to the user's perspective for transmission.Due to the delay,the user's perspective position probably changes when the user views this video segment,so it is difficult to guarantee the quality of the video in the viewing area.Therefore,we adopt the viewport prediction technique in the viewpoint-adaptive transmission,and propose a joint prediction method of path information and content features.We introduce the principles of linear regression,back propagation network and long short term memory network,and contrast their performances in accuracy.Experiments show that the joint prediction method can more effectively predict the future perspective position,and greatly improve the terminal picture quality in the actual delay system.