Passive Non-Line-of-Sight Imaging Methods Based On Deep Learning

Passive non-line-of-sight(NLOS)imaging method can reconstruct 2D representation of the hidden scene using an ordinary camera.In the field of petroleum engineering,this method can be applied in oil pipeline inspection,rock crack detection,and offshore platform safety monitoring.The most successful passive imaging methods currently rely on adding extra penumbra to the imaging scene.The method has two shortages: on the one hand,the imaging algorithm has low restoration accuracy,slow speed,and is susceptible to interference;on the other hand,the method requires the penumbra condition,but in most scenarios such requirement cannot be satisfied.Addressing the above issues,firstly,this paper studies the imaging scene with penumbra,and proposes deep learning models to replace the traditional algorithm;then,this paper studies the scene without penumbra and proposes a prior-based model.The details are as follows:In a scenario which contains additional penumbra,this paper proposes two imaging models,and gives a comparative analysis.The first model is a direct encoder-decoder imaging model.The encoder-decoder framework can directly learns the mapping of observed images and hidden scenes through an end-to-end training method.The method is simple and efficient while requires zero knowledge of the modeling scene.The second model is the constrained encoder-decoder model.By introduces the light transport matrix as a constraint of the model,this method avoids to use decoder directly for reconstruction which gives model a better generalization ability.Compared with the traditional method our methods have following advantages:(1)In terms of imaging results,the two models were tested on the handwritten digit dataset and the mobile handwritten digit dataset.The results show that the both model has higher restoration accuracy and faster imaging.(2)In terms of robustness,by testing the model in three different levels of ambient light,it is proved that the model has strong resistance to ambient light.In a penumbra-free scenario,this paper proposes an imaging model based on prior knowledge.In our method,the prior knowledge is introduced to make up for the missing imaging information in the penumbra-free scene.The model uses the discrete variational auto encoder to encode the prior knowledge and the observation to generate corresponding latent variables,then these variables are feed to a Transformer block so that the Transformer blocks can learn the connection between the prior knowledge and the observation,and finally the decoder completes the imaging process by decoding output of the Transformer.In the experiment,the model is tested under different difficulty datasets,and the results show that the model has excellent imaging accuracy and strong robustness.Experiments also test the role of attention mechanisms and prior knowledge in the structure,demonstrating the effectiveness of the model structure and explaining how the model utilizes prior knowledge to accomplish imaging.