Research On Projector Depth-variation Out-of-focus Mechanism And Elimination

With the development and promotion of augmented reality and human-computer interaction technology,projection display technology has being applied to more and more unconventional projection display environments to achieve the needs of immersive experiences.In the space augmented reality application scenario,due to the characteristics of the optical structure of the projector itself,when projecting a complex surface or a dynamic surface,non-uniform defocus blur will appear,affecting display quality,thereby reducing the immersion of the augmented reality.Existing projection defocus blur estimation and elimination methods have the following limitations: a.Non-parametric simulation and elimination techniques achieve better accuracy,but do not have the ability to simulate and eliminate variable depth scenes;b.Parametric simulation and elimination technology has strong a priori and insufficient model expression ability,whose simulation and elimination performance for defocus blur is limited.This paper utilizes deep learning technology to propose a variable depth projection defocus blur estimation and elimination algorithm based on multi-source information fusion to solve the limitations of existing methods.First,an information acquisition system equipped with a variety of image sensors is designed and calibrated.Through this system and corresponding processing algorithm,a highly accurate multi-source projection defocus dataset is constructed.Then,utilizing deep learning technology,a multi-channel deep convolutional neural network model based on residual structure is constructed,and a variable depth projection defocus estimation algorithm based on multi-source information fusion is proposed,which overcomes the bottlenecks of traditional parametric and non-parametric algorithms.Finally,an improved accelerated convolutional neural network is further constructed,and a fast variable depth projection defocus blur reduction algorithm based on a lightweight network is proposed to achieve fast defocus elimination compensation image calculation without parameters adjustment and iterative optimization.The main research contents are as follows:(1)Analyze the research status of projection blur removal technology,summarize the main difficulties of projection blur removal technology,and present the research content and overall structure of this paper.(2)Design a multi-source image data acquisition system and give the hardware selection;then,complete the intrinsic parameter calibration of the Kinect V2 visible light camera,Kinect V2 depth camera and projector;finally,utilize the "camera-projector" system calibration method and transformation matrix to complete the calibration of relative extrinsic parameters between various devices.(3)Research on the acquisition strategy and processing algorithm designed for multi-source image dataset including depth-focus/defocus images.Firstly,the overall framework of the data acquisition strategy and processing algorithm is designed;the global distortion correction and preliminary alignment of multi-source image information are achieved through spatial transformation and higher-order geometric polynomial alignment techniques;based on the translation error loss function,the relative translation between focus and defocus image is finetuned frame-by-frame;the brightness change factor of the focus and defocus image is estimated to achieve the brightness alignment;finally,the non-uniform brightness correction of the projected image is achieved through the reference image.(4)Research on depth-variation projection defocus blur estimation technology based on multi-source information fusion.Firstly,analyze the mechanism of projection defocus blur;construct a multi-channel deep convolutional neural network model based on residual structure,and propose a variable depth defocus blur estimation algorithm based on multi-source information fusion,overcoming the limitations of the existing method,achieving efficient fusion of multisource information,improving convergence and estimation accuracy.Finally,experiments have verified the effectiveness of the algorithm proposed in this paper,achieving better blur removal effect.(5)Research on fast depth-variation projection defocus blur elimination technology.Firstly,the existing blurring removal technology and methods are studied;a lightweight and fast multichannel deep convolutional neural network model is constructed,and thereby a variable depth projection defocus blurring elimination algorithm based on a lightweight network is proposed which achieves much faster defocus elimination with equivalent performance.Finally,experiments verify the feasibility and effectiveness of the proposed algorithm.