Research On Real-time Coding And Rendering Method Of Three-dimensional Light Field Display

Real-time display system can provide dynamic and interactive images that plays an important role in human production and daily life.The emerging three-dimensional(3D)light field display technology can present full-parallax and continuous-viewing 3D image without convergence-accommodation conflict and any auxiliary devices,which significantly enhance the viewing experience and understanding of 3D scene.Therefore,it is widely concerned by domestic and foreign research institutions as the most potential way of display in the future.In corresponding to the display technology,rendering technology can provide photorealistic image,greatly expanding the application of the display system.The imaging mechanism of 3D light field display is different from that of two-dimensional(2D)display,so the 2D rendering methods are not suitable for 3D light field display.Traditional 3D light field display adopts sequently views rendering strategy before light field coding,which has the shortages of high complexity and redundancy.For super multi-view based 3D light field display,it can not achieve fast and efficient light field coding and rendering.In addition,the 3D light field display system with large viewing angle and high resolution can provide better immersion viewing experience.However,increasing the viewing angle of 3D display will inevitably rendering more viewpoints;and enlarging the resolution will also increase the content of each viewpoint.As a result,the rendering time also increases.Aiming at the improvements of 3D light field display on rendering efficiency and display quality,this dissertation studies the method of fast light field coding and rendering.The innovation and main research contents are as followsResearch point 1:Real time 3D light field display technology based on volume rendering.To real-time display the spatial information of volume data on 3D light field display,this dissertation proposes a modified single-pass multi-view rendering method.By modifying the volume rendering pipeline,the mapping of the synthetic image pixels and multi-perspective informations are directly accomplished in once rendering pass.This pipeline can convert 3D volume data into high-quality 3D image on 3D light field display devices.The whole process is completed in parallel on Graphics Processing Unit(GPU),and there is no redundancy computation.The rendering frame rate of multi-view synthetic image is more than 60 fps at 4K(3840×2160)resolution pixel-level,and the subpixel-level rendering frame rate is more than 30fps.Research point 2:Real-time coding and rendering technology of splicing and ultra high resolution light field display based on rasterization rendering.To realize real-time rendering and display of large-scale 3D image,this dissertation constructs a 3D light field display system with super-high resolution and super-large size based on multi screen splicing technology.A distributed 3D light field rendering scheme with real-time,super-large size and super-high resolution is realized,which is composed of a host and several clients.User datagram protocol(UDP)is used for the communication between the host and the devices.The host is responsible for the packing and sending of the control signals,the supervisory of the clients running status and the files transmission.The clients receive the control signal from the host,and rendering the light field animation with that signal.Each client adoptes multi-view rendering scheme based on rasterization and only rendering the sub-synthetic image of the light field sub-element driven by it.To dispose the stitching between multi-screens,texture post-processing is applied to the pixels in the stitching area of the output image to ensure the continuity of whole 3D image.A display system composed of 15 light field sub-elements with 23040×21600 rendering resolution is realized,real-time rendering frame rate is over 40 fps,and the delay of network and rendering are stable at 30 ms,ensuring the quality of 3D image and users'interactive experience.Research point 3:Real-time coding and generation of ultra high-quality 3D image based on physical based rendering and RCNNs post-processing denoising.To generate a high-realistic 3D image with physical illumination,physical based rendering(PBR)technology is adopted.Path tracing based Monte Carlo integral can realize PBR in GPU.According to the optical layer structure of the 3D light field display,pixels are coded to specific viewpoints,then the directional rays are casted from viewpoints to pixels direction for ray path tracing.In the process of Monte Carlo integral,advanced illumination is used for high-quality elemental image array(EIA)generation.Directional rays are recursively calculated according to the Bidirecitonal Reflectance Distribution Function(BRDF)material.The experimental results prove that at 4K(3840×2160)resolution,2 samples/pixel real-time frame rate exceeds 30 fps.As a result,the proposed method is experimentally demonstrated to generate 3D image with global illumination.Ray path tracing algorithm based on PBR requires extremely high sampling rate to achieve high realistic light field rendering.3D image that have not yet fully converged during MC integral exhibits noises,which can be eliminated by mass of samplings.The existence of noises seriously affects the viewing experience.To quickly generate high-quality 3D image,a pre-trained RCNNs denoising filter is introduced to filter these noises.The Monte Carlo integral results,albedo,normal and material are used as the input of RCNNs denoising filter to construct the optimal filter structural parameters.With continuously sampling,integration and denoising,our method can achieve high-quality 3D image in a very short time.Structural similarity metric(SSIM)value and Peak Signal to Noise Ratio(PSNR)gain of the reconstructed 3D image and target 3D image exceed 90%and 10 dB within 10 frames,respectively,showing the super capability of our method in 3D image generation.