High-precision SVBRDF Recovery Based On Images

In physics-based rendering research work,high-precision material modeling is an important part of realistic rendering techniques.With the wide application of realistic rendering technology in film and television,games,industrial production and other fields,the industry’s demand for rendering realism is getting higher and higher,giving rise to the research boom of highly realistic material modeling.With the advantages of small storage space and easy editing,SVBRDF(Spatially Varying Bidirectional Reflectance Distribution Function)is widely used for realistic sense material representation.The most intuitive way of material modeling is to collect high-dimensional observation samples using special acquisition devices,but due to the drawbacks of high acquisition cost and low storage and processing efficiency,researchers have turned to imagebased material reconstruction work.Reconstructing the surface reflectance of an object from a small number of images is a highly discomforting problem,and researchers have achieved material reconstruction from static images using optimization or hypothesis methods.With the development of neural networks and other technologies,researchers proposed a deep learningbased material reconstruction method,which further extended the application of image-based material reconstruction work and achieved SVBRDF reconstruction using one or a small number of images,but the reconstruction network performed poorly on some materials such as highlight materials and time-varying materials,and the reconstruction results showed highlight artifacts and temporal inconsistency.In this paper,we propose a single-frame SVBRDF reconstruction method based on highlight removal module.For artifacts and overdarkness caused by highlight regions,this paper proposes a conditional generation adversarial network with a highlight removal module and designs a joint loss function.The core of the network framework is an adversarial multidiscriminator network with a highlight removal module.By using a multilevel identification highlight removal module with dense feature fusion connections,the input image can be converted to a highlight-free image to further extract more meaningful features in the highlight region.It is demonstrated by quantitative analysis and qualitative visualization analysis that the method proposed in this paper can reconstruct high-quality diffuse mapping,normal mapping,roughness mapping and reflection mapping from a single image with highlights,and the generated mapping can render more details in overexposed regions and the rendered image in the new viewpoint is more consistent with the input image.In this paper,we propose a continuous-frame SVBRDF reconstruction method based on the time domain,which is modified on the basis of the single-frame reconstruction network by introducing a reweighting module.The key idea is to learn the correlation between the feature information of the input image and the corresponding moment index,to continuously use the distribution of the temporal index to influence the distribution of the features,and to establish an implicit relationship between the temporal index and the features.In addition,inter-frame loss is introduced to measure the temporal consistency between adjacent frames,and different weights are given to the reconstruction loss of each frame to ensure the spatial quality of the material features corresponding to the input image as much as possible.It enables the network to reconstruct the SVBRDF mapping at any moment,thus enabling reconstruction of consecutive frames.In summary,this paper achieves results on single-frame reconstruction that is able to produce temporally continuous material mapping on continuous-frame material reconstruction,and the recovered single-frame and continuous-frame material re-rendered images are able to maintain visual consistency with the input image.The material maps reconstructed using this paper have good performance in rendering real scenes and can be used as material resources by other rendering engines to render detailed and realistic rendered images.