Study Of Multispectral Light Field Imaging System And Algorithm

The seven-dimensional plenoptic function is proposed by E.H.Adelson as the holographic expression of the visual world,it describes the process of incident light at wavelength λ entering human eyes whose view position is x,y,z with the direction ofθ,φ at time t in the 3D space,and the proposal of the function makes it possible in plenoptic reconstruction.On the basis of the plenoptic function,light-field camera recording light ray positions and spectral camera recording light wavelength both appear,but the state-of-the-art imaging systems still face two main problems:the lost of depth information z and spectral dimension X.Actually,current imaging apparatus can only acquire two-dimensional spatial information and loses the spectral and depth information,which play very important roles in many reality applications,such as material discrimination and classification,object tracking and 3D reconstruction et al.The challenges of capturing high-quality light-field and spectral information simultaneously and efficiently are getting more and more attention.The main contribution of this dissertation is the combination of the light-field theory and spectrum multiplexing theory.We introduce a framework for affordable and easy-to-use multispectral light field imaging using heterogeneous cameras array system,and the whole system consists of three main subsystems:heterogeneous cameras array,processing boards and a PC.Camera array with wideband filters in front of it are mounted parallelly and each camera is connected to a processing board for system triggering and parameter configuration,finally the data is sent to the PC for further application.A stereo matching algorithm using convolutional neural network is developed to compute the correspondences between the heterogeneous multi-view images,and the resulted disparity map is used to calculate the depth information of the real scene.The proposed spectral de-multiplexing algorithm is applied to reconstruct the multispectral information that tripling the number of cameras,and thus prove the simultaneous acquisition of both light field and spectra of the scenario.Our system prototype can be further used in video capturing.Different from the light-field camera consist of micro-lens array,there is no need for multi-camera arrays to gain spectral/angle resolution by sacrificing space resolution,which promise the acquisition of high space resolution and high spectral/angle resolution at the same time.This dissertation uses Autodesk-3ds simulated data and standard database to test and analyze the error of the proposed system algorithm,which gain great evaluation result and prove the system’s reproducibility and robust,we also provide real captured data to prove the feasibility of the system in real world application.