Research On Infrared Video Colorization Algorithm Based On GAN

The infrared imaging technology has the ability to capture images that are not detectable by the naked eye or visible light detectors in low light or extreme environments based on its unique thermal imaging principle.However,the grayscale images generated by infrared imaging do not align with human visual perception habits,lacking many details and texture information,making them difficult to interpret and analyze directly.In practical applications,the unsupervised conversion from the infrared domain to the visible light domain in video has a wider range of application value in various fields.In view of the existing shortcomings of the deep neural networks used to achieve infrared video colorization,such as complex structures,unstable effects,inter-frame flickering,and slow generation speed of the generator,this thesis proposes a new infrared video colorization algorithm based on Generative Adversarial Networks(GAN).The main contributions of this thesis are as follows:1.Improved the generator and discriminator network structures in the GAN model to enhance the colorization effect of video frames.Self-attention layers are introduced into the original generator to capture global information in the input data,better understanding the dependencies between input data,and helping the generator synthesize visually more realistic color images.The original discriminator network structure is improved by using a multi-scale discriminator to discriminate different-sized image blocks,which improves the discriminator’s discrimination accuracy.The effectiveness of the improved generator and discriminator structures is verified through ablation experiments and comparative experiments.2.The concept of contrastive learning is introduced to ensure consistency of semantic information between generated samples and original samples.By maximizing the mutual information between corresponding image blocks in the input image and the target domain image,the model focuses on the content of the image rather than external appearance.Compared to the strict bi-directional mapping brought by mainstream cycle consistency methods,our approach reduces the complexity of the unsupervised model structure and stabilizes the generated image results.3.Proposed a video frame consistency method to address inter-frame flickering.The differences between input video frames and generated video frames are constrained in different feature spaces,utilizing the constraints of temporal consistency loss function to train the generator to generate video frames with consistent inter-frame differences.The frame consistency method is validated through subjective analysis in comparative experiments,showing that it can produce smoother results in infrared video colorization.4.Improved the GAN compression framework.In this thesis,the trained GAN model is compressed and the existing GAN compression methods are improved by introducing channel attention mechanism into the distillation network and adding perceptual loss between the images generated by the teacher generator and the student generator.The improved GAN compression framework is validated through comparative experiments,showing that it can improve the generation speed of the model while maintaining the visual effects of colorization.