Research On End-to-End Image Coding For Visual Characteristics

With the rapid development of computer vision technologies,a large amount of data is transmitted and stored for human or machine perception.To reduce storage and transmission pressures,joint perception coding frameworks for human and machine vision have been extensively studied.Compared with traditional coding methods focused on human vision,human-machine perception coding encodes images according to the different characteristics of human and machine vision,which can reduce data bits without affecting image perception quality.This thesis focuses on the research direction of human and machine perception coding and explores in depth the frequency domain perception characteristics of classifiers,multimachine vision task Just Noticeable Difference(JND),and perception coding optimization schemes based on deep learning.The main innovative research results of this thesis include:(1)This thesis develops an algorithm to analyze the visual classification model based on frequency domain perception characteristics.The current research on the operation mechanism of deep neural networks is based on the pixel domain and lacks the exploration of the frequency domain.Since the frequency domain is a powerful tool for image processing and has good interpretability,this thesis proceeds to a preliminary attempt to interpret the deep learned classifier in the frequency domain,trying to find the correlation between the subbands of the input image and the results of the deep learnt classifier.The experimental results show that the results of the deep classification model depend on specific subbands and their correlation coefficients.(2)This thesis develops an algorithm for multitask machine vision Just Noticeable Difference(JND).This thesis proposes a multi-task JND model based on the current lack of a universal and accurate optimization objective for machine vision coding.The JND model is then optimized under multiple visual task constraints to produce the largest possible JND threshold.The experimental results show that this algorithm can inject JND noise into the original image to achieve a PSNR of 16 d B without affecting the results of multiple visual tasks.(3)This thesis develops a JND-based Perceptual Optimization for Learned Image Compression.The lack of efficient perception optimization schemes is the main reason for the low perception quality of deep-coded images,and based on this,this thesis proposes a perception loss function based on JND to introduce human vision characteristics into the model optimization process.The JND level is adjusted according to the distortion level of the reconstructed image during the optimization process.Experimental results show that this scheme can significantly improve visual perception quality at the same bit rate.