Biological Visual Cognitive Mechanisms Inspired Research On Road Scene Thermal Image Colorization Using Deep Networks

Nowadays,technologies of driver assistance and automatic driving systems are booming.Infrared imaging has good application prospects in driving scenarios because of its high robustness to environment,weather,and light.However,infrared images lack color information and spatial resolution.Due that color is much more distinguishable than grayscale for human,it would be helpful for people to perceive and understand the driving scenes if infrared images can be transformed to visible images.At the same time,most existing computer vision researches for road scenes train the deep networks using visible images.After transforming infrared images to visible images,we can directly use those pre-trained networks for subsequent segmentation,detection and other tasks.Generative Adversarial Networks(GANs)have achieved rapid development in recent years.GANs learn the distributions of real data through adversarial training between generator and discriminator,so as to generate as real results as possible.Image translation is an important application of GANs.To translate infrared image to visible image,we need to keep its structure and content unchanged while adding reasonable color and texture information to it.In this thesis,inspired by biological visual perceptual mechanisms,we improve GANs from two different ways and propose a Gestalt visual perception inspired infrared image colorization algorithm and a hierarchical visual perception inspired infrared image colorization algorithm.The details of these two works are as follows:1.The Gestalt visual perception inspired infrared image colorization algorithm.Our algorithm is inspired by the Gestalt theory of visual perception.We use the non-local attention mechanism to build the global connection between the corresponding images so as to transfer the visible image’s color to the infrared image.What’s more,we make use of deep networks’ features and adopt loss functions including contextual loss,perception loss and contrastive loss to constrain the structure,content and style of the generated image.2.The hierarchical visual perception inspired infrared image colorization algorithm.Learning from the hierarchical information processing of viual system,we adopt a hierarchical structure to serially process the colorization of thermal image.The original task is decomposed into two independent sub-tasks,i.e.,transforming infrared image to grayscale image and colorizing grayscale image.The first sub-task is mainly responsible for dealing with spatial non-correspondence,while the second sub-task is responsible for generation of reasonable color information.Experiments of both subjective judgements and quantitative comparisons show that our visual perception mechanism inspired algorithms mentioned above have achieved better colorization performance than representative existing methods.