Research On Visual Perception-oriented Light Field Image Quality Assessment

With the rapid development of visual media,Light field(LF)imaging has received significant attentions from both academia and industry,which allows a wide range of applications in the areas of computer vision and computer graphics,such as three-dimensional reconstruction,virtual reality,image-based rendering,to name a few.Unlike the traditional imaging technology which directly registers the intensity of light rays on the camera sensor,the LF imaging aims to record the intensity of light rays of a scene point in different directions at the sensor plane separately.Thus,an LF image essentially records not only the color information of the scene but also the underlying depth information.As a type of digital visual signals,technical challenges will be inevitably encountered across various processing stages—from acquisition,encoding,denoising,transmission,rendering,to display.The distortion incurred in each stage will affect the perceived quality of the processed LF image and related system performance.It is quite desirable to develop the IQA methodology for automatically and accurately measuring the perceptual quality of the LF images.This thesis analyzes the characteristics of human visual system(HVS)and LF images,and proposes three LF image quality assessment(IQA)models.The specific descriptions are as follows:1.Considering that a set of sub-aperture images(SAIs)captured in the same scene only have a slight difference in the viewing point,but have quite similar contents to each other.Thus,it could be logical to consider more image details for distinguish the difference when extracting image feature information in the design of IQA model for the SAIs.Thus,we propose amulti-order derivative feature-based LF IQA model(MDFM),and use multi-order derivative features to effectively explore the image details.2.Note that an LF image records not only the two-dimension information of the scene but also the underlying three-dimension information.We propose a the symmetry and depth feature-based model(SDFM).The proposed SDFM uses the symmetry and depth feature to explore the special characteristics of an LF image.Note that they reflect the quality variation of the structure information and geometry information of an LF image,respectively.3.Based on the fact that to compare two LF images,it essentially boils down to measure how coherent of these two LF images.We propose a novel IQA model for LF image,called the log-Gabor feature-based light field coherence(LFC).This model extracts the multi-scale log-Gabor features from the SAIs and the single-scale log-Gabor feature from the epi-polar images(EPIs),respectively SAIs and EPIs.Note that the former can reflect the image details,the latter indicates the viewing consistency.In summary,the proposed IQA models for LF images in this thesis have a better ability to be consistent with the perception of the HVS on the evaluation of the LF image.Experimental results of MDFM,SDFM and LFC have certain theoretical research significance and practical application value for LF IQA and of related algorithms optimization of LF image processing.