Synthetic Aperture Photography Based On Moving Camera

As a novel visual information sensing technology,synthetic aperture photography(SAP)can be utilized to obtain the see-through imaging results of occluded objects via only visible spectral sensors.The technology could enhance the performance of object sensing when the objects are severely occluded or the background in the scene is complex and shows its important prospects in security surveillance,military and other fields.In conventional SAP approaches,the baseline of the visual system and the perspectives of the viewpoint are fixed.Besides,no refraction is assumed in conventional SAP approaches.In this thesis,we mainly focus on two problem that are related with SAP,the mobile and unstructured capturing of light field data and SAP in multiple refractive media scenes.The proposed approaches in the thesis could extend the application environments of SAP,such as car based or aerial based photography,while expanding the dimensions and capabilities of visual information perception in multiple refractive media scenes.Recent SAP approaches and related techniques are comprehensively surveyed and ana-lyzed.We summarize the mathematical model,main features of SAP and several key techniques including imaging parameter estimation,focal surface estimation and the visibility analysis of the imaging scene,which would provide valuable references for future research works.The main work and contributions are summarized as follows:(1)A novel SAP approach based on a mobile camera-inertial measurement unit system is proposed.By moving the system on a planar surface,a virtual array of sample points can be simulated.With geometric property assumptions,transformation matrices for synthetic aperture image generation can be factorized into two parts and can be estimated separately via inertial and visual data.The inertial data based estimation reduces computational loads in the entire es-timation.Compared to the conventional approaches,the proposed method effectively improves the solution for synthetic aperture images capturing and generation by using a more flexible,convenient and inexpensive system.(2)A multiple moving cameras based collaborative SAP framework is proposed.In the proposed framework,moving cameras are divided into two types:one main camera and several sub cameras.Based on the sparsely reconstructed 3D map of the scene from the main cam-era,the geometric relations between the main cameras and the sub cameras,which provide the unstructured and sparse sample viewpoints,are established.The proposed framework could increase the flexibility in unstructured light field data capturing,while complementary infor-mation from multiple sub cameras could improve the see-through performance in the synthetic images.(3)A multiple view constraint based refractive imaging parameters estimation method is proposed.With the assumption of physically correct refractive model,the imaging parameters in the multiple refractive media scene are analyzed and estimated based on the multi-view con-straint Compared to recent approaches,less a priori information is needed in the proposed method.A refracted ray representation is presented and utilized to generated the refractive SAP results,Quantitative experiments show that the parameters estimated from the proposed method gives better accuracy and imaging results than the current single view based approach.(4)A novel refractive SAP approach based on a linearly translating camera is proposed.Based on the geometric assumptions of the capturing device in our system,the time-varying housing parameters and the camera positions are simultaneously estimated online.With the assumption of multiple layer flat refractive geometry,while the number of refractive media layers is not spcifically defined,the refractive SAP approach in multiple refractive media and occluded scenes is proposed.Quantitative experimental results show that the SAP results can not only see-through occlusion,but also are free of refractive distortion.