| As a new form of immersive media,the Light Field(LF)with dense viewpoints can provide users with a true 6 Degrees-of-Freedom(DoF)immersive experience,because dense LF captures the scene with photo-realism,including aperture-limited changes in viewpoint.Nevertheless,the larger size and higher dimension of dense LF video data bring greater challenges to processing and transmission.This research optimizes the architecture of dense LF video services from four aspects:data compression,transmission optimization,data mining and communication computing collaborative improvement.In terms of dense LF data compression,uncompressed dense LF video data are too large for network transmission,which is why dense LF compression has become an important research topic in recent years.A new dense LF compression algorithm based on Graph Neural Network(GNN)is proposed in this work.It can use the graph network model to fit the similarity between the LF viewpoints,so that only the data of a few essential anchor viewpoints need to be transmitted after compression,and a complete LF matrix can be reconstructed according to the graph model at the decoding end.This method also solves the problem of weak generalization of the LF reconstruction algorithm when dealing with high-frequency components through the design of two-layer compression structure.Compared with existing compression methods,a higher compression ratio and better quality can be achieved using this algorithm.For the dense LF transmission optimization,to improve the adaptability of the realtime requirements of different dense LF applications and robustness requirements in unreliable network environments,an adaptive dense LF video transmission scheme based on Multiple Description Coding(MDC)is proposed.It can divide the LF matrix into LF descriptions at different levels of downsampling ratios,and optimize the scheduling of the descriptions transmission queue,which can ensure that it can adaptively adjust the design of basic GNN unit so that the proposed method can adapt more flexibly to the real-time changes of user viewpoint requests,so as to save unnecessary viewpoint transmission overhead to the greatest extent,and minimize the adverse impact of network packet loss and network status fluctuations on LF transmission services.For dense LF data mining,depth estimation has been a very hot topic in recent years.To achieve a good balance between the performance of both narrow-or wide-baseline dense LF data,a optical-flow-based LF depth estimation scheme,which uses a convolutional neural network(CNN)to predict the patch matrix after optical-flow-assisted offset,is proposed.After the opticalflow-assisted offset,the disparity between patches is processed to a unified numerical range,which can effectively solve the overfitting problem of the LF depth estimation network caused by the uneven distribution of the baseline range of LF samples.Experimental results show that the proposed patchcompensation-based estimation mechanism has good generalization on dense LF data of different baselines and is compatible with various existing narrowbaseline LF depth estimation algorithms.Finally,for the collaborative improvement of communication and computing,since dense LF depth estimation and other processing place high requirements on both the computing and caching capabilities of the infrastructure,a system model that combines Multi-access Edge Computing(MEC)technology with dense LF video services is proposed in this thesis.In this study,the problem is transformed by the Lyapunov optimization,and an optimized search algorithm based on the Markov approximation method is designed,which can adaptively schedule and adjust the task offloading strategy and resource allocation scheme,so as to provide users with the best service experience in the LF depth estimation task.Numerical results demonstrate that this edge-based model can achieve a dynamic optimal balance between energy and caching consumption while meeting the low latency requirements of dense LF video services. |
PDF Full Text Request