Light Field Super Resolution Of Cross-scale Via Zero-shot Learning

In recent years,light field imaging technology has aroused wide concern from researchers in computational photography,computer vision,and optics.Due to the spatial-angle resolution trade-off,the problem of insufficient resolution of light field imaging still prevents the further development of light field system applications.Comparing to traditional image super-resolution,the deep learning based image super-resolution technology has achieved the state-of-the-arts super-resolution results in recent years.However,the learning or training process of this kind of scheme relies heavily on the training datasets,which leads the insufficient generalization performance in its utilizations.This paper proposes a new light field spatial-angle domain super-resolution reconstruction algorithm based on zero-shot learning algorithm.Different from the traditional model with a clear scene prior,this paper proposes to learn cross-scale reusable features in the light field eipolar plane image space.Without using any external training datasets,the algorithm simultaneously performs super-resolution reconstruction in both the light field spatial and angular domain.On multiple public light field datasets,the proposed algorithm has achieved significantly better image super-resolution resultsthan classic deep learning algorithms.Althoughour results does not outperform the existing light field super-resolution method based on large sample learning,the proposed algorithm can be comparable to the state-of-the-arts in angular domain super-resolution performance.The main research work of this paper as follows:(1)According to the zero-shot learning theory,this paper proposes to combine the zero-shot learning with the super-resolution reconstruction of light field images by constructing a light weight CNN network.Without any external datasets,the network can conduct the cross-scale learning using an input light field and its down-sampled counterpart for training.After training,the final prediction and reconstruction of the high-resolution light field can be applied on the original scale that generates a super-resolved light field in both spatial and angular domain.(2)This paper further uses a deep residual network to super-resolve the light field.On the basis of the above zero-shot learning reconstruction,we have optimized the proposed network using a residual module based spatial domain reconstruction,which enhances the high-frequency details.The reconstruction data results show that the algorithm effectively improves the reconstruction precision and calculation efficiency.The results are significantly better than the other methods in quantitative comparisons,such as peak signal-to-noise ratio,structural similarity,and running time.(3)To realize the software development,this paper employ the PyQt framework to build an UI interface of the above super-resolutions.This paper shows a series of software development,which includes software analysis,design,development and functional testing.All the functions of this software can meet the design expectations in the final testing.It provides a visual solution for users to operate the proposed light field super-resolution reconstruction.In summary,this paper analyzes the cross-scale internal similarity of the light field,and proposes a light field super-resolution algorithm,which does not need any external dataset.By extracings cross-scale reusable features in the EPI space,the proposed algorithm can simultaneously perform super-resolution reconstruction in both spatial and angular domain of a given light field.