Research On Image Restoration Algorithm Using Cross-Modal Guidance

With the increasing development of multi-sensor devices,scenes can be saved as multimodal digital images at the same time,such as visual images,depth images,etc.These images are related in content,but the image quality is different.The image restoration algorithm using cross-modal guidance takes high-quality images as guide images,which provide reference information of scene structures for low-quality target images and assist the restoration process of target images.However,the existing algorithms of image restoration using guidance require the alignment of the guide image and the target image,which is difficult to directly handle scenes of dual cameras.At the same time,the performance of cross-modal guided image denoising is poor,making it difficult to put into application.Therefore,we research the restoration algorithm using cross-modal guidance from the following two aspects:1.We build a guided restoration framework based on the parallax attention mechanism,and propose the guided image denoising and demosaicing algorithm for monochrome-color binocular cameras.Given the imaging characteristics of binocular cameras,the proposed algorithm generates high-quality aligned guide images based on the parallax attention mechanism and introduces the information from the monochrome camera into the denoising and demosaicing process of the color camera,which has achieved a significant improvement for the restoration results compared with the comparison algorithms.2.We propose a cross-modal guided denoising algorithm based on frequency decomposition prior,and build a frequency-related residual learning network to improve the cross-modal guided denoising performance.The network makes full use of the frequency domain properties of the noise to efficiently smooth the noise.At the same time,based on the correlation of multimodal images in the frequency domain,accurate structure transfer is achieved.Relying on an explicit frequency domain decomposition framework,the network balances the two tasks of noise removal and structure reconstruction well.On three cross-modal guided denoising tasks,the proposed algorithm significantly improves both denoising accuracy and visual quality.In short,we extend the algorithm of image restoration using cross-modal guidance to scenes of dual cameras by introducing the parallax attention mechanism and effectively improve the performance of the cross-modal guided denoising algorithm by introducing the frequency decomposition prior.