Research On Multiple Description Image Coding Based On Semantic Segmentation

With the rise and application of smart devices such as smart cities and smart factories,more and more data need to be stored and transmitted.Despite the increasing bandwidth and computing power,scenarios such as Internet and wireless links are forced to make a trade-off between quality guarantee and resource utilization.There are still some risks of transmission failures,when the Internet congestion occurs in the overloaded case or signal packets are conveyed in the unpredictable and unreliable channels.In order to solve these problems,multiple description coding(MDC)has been studied.Semantic segmentation can effectively extract information of image pixel level,and semantic segmentation map can significantly improve image reconstruction quality in image synthesis or other image processing tasks.Therefore,this paper utilizes semantic segmentation label as auxiliary information to research on deep neural networks-based multiple description coding.The main contents of this paper are as follows:(1)Deep semantic segmentation-based multiple description coding method is proposed.As is well known,the image is composed of many pixels.And semantic segmentation needs to classify pixels according to their different semantics in the image,divide the image into different regions and recognize the content of the region,and use different tags to represent different objects of the image.Making use of semantic information when encoding images means better use of image features.And most of current compression methods only convert the images at pixel level without consider the semantic information.This paper improves deep semantic segmentation-based image compression,and proposes a deep semantic segmentation-based multiple description coding method to improve image encoding efficiency and enhance the robustness of transmission.The experimental results show the scheme is better than the standard image compression methods.(2)Multiple description coding network based on generative adversarial network is proposed.The proposed network is based on the deep neural network,which can extract the image's highlevel features according to the semantic information of the image.Firstly,the whole framework includes multiple description generator feature network that make use of image features to generate multiple descriptions,multiple description reconstruction networks based on generative adversarial network(GAN)for reconstructing image.Secondly,semantic segmentation map is used as auxiliary side information in the whole network.As label information,semantic segmentation map plays an important role in generating and reconstructing images.Testing on more standard datasets,the experimental results prove the effectiveness of the proposed multiple description coding network.