Study On Image Deblocking,Multi-Spectral Fusion And NIR Colorization Using Convolutional Neural Networks

With the development of media technology and information networks,digital TV vision systems and display devices require images with high quality.A large amount of raw data can be easily transferred and stored after being compressed.However,high compression rate is inevitably accompanied by information loss.The decoded images at low bit rate produce serious blurring and blocking artifacts,which greatly decreases the image quality.Image post-processing is a key technique for removing artifacts from compression.In photographic imaging,with the development of sensing systems,near-infrared(NIR)images have been widely used.Multi-spectral image fusion under low light conditions has become a viable solution for various application scenarios,such as night scene photography and security monitoring.Visible light camera devices in low light are subject to noise,and the NIR image in the same scene can provide details from different spectral images.To obtain high-quality fusion results,researchers have carried out study on multi-spectral image fusion algorithms.When regular cameras hardly provide information in the dark environment,only NIR image is able to provide visual aid.NIR image decreases the user acceptance because gray-scale image is not agree with human cognition.It is practical to colorize NIR image considering human perception.With the rapid development of computer science,deep learning has shown great power in the field of image processing.Based on the related research work,we focus on image deblocking,multispectral fusion and NIR colorization in this thesis.The main work of this thesis is as follows:1.A gradient-guided compressed image deblocking algorithm is proposed.Block-based independent coding results in a blockiness of the compressed image,and since the correla-tion between the coding blocks is not considered,the gradient of the image at the edge of the block produces a significant change.The gradient map accurately locates the edges of the block.Previous work has focused on restoring the construction of the network without con-sidering the priori information of the compressed image.We concurrently recover the image gradient during the image restoration process.The restoration of images and gradients is op-erated by two sub-networks.The features generated in the gradient recovery is fused to the image inference sub-network to guide the recovery of the compressed image.Image restora-tion is closely guided by the associated gradient features and produce noise-free output with fine details.2.A multispectral image fusion algorithm based on convolutional neural network is pro-posed.Different from traditional image fusion algorithms,we successfully apply convolu-tional neural networks to the fusion of NIR images and visible images.Due to the difficulty of capturing different data sets for different lighting scenes,we synthesize the available train-ing sets using the data set under normal lighting conditions.We build a two-stage network for denoising and fusion respectively,multi-level aggregation architecture is adopted.To obtain a high-quality fused results with the original color of the visible image and the rich details of the NIR image,we propose loss functions consists of two terms:the perceptual loss function and the multispectral loss function.With the optimal weight of the two loss terms,the fused image removes the noise under low illumination,and correctly preserve the original image color,while containing rich texture information.3.A NIR colorization algorithm based on convolutional neural networks is proposed.The NIR image colorization is a challenging problem.Although it is similar to traditional grayscale colorization methods,it needs to overcome the discrepancy between multi-spectral image and single-channel brightness image,and recover three-channel color image.We use a model that can effectively utilize multi-scale information as a basic convolution unit to successfully recover the color of NIR images.We propose to use structural similarity as a loss function.To further optimize the coloring results,we constrain the output in the gradi-ent domain to remove the noisy texture of the flat region.The experimental results show that the proposed NIR colorization architecture has good performance,and the proposed loss function can be successfully applied to the coloring problem of NIR.