An End-to-end Panoptic Segmentation Algorithm Based On Encoder-decoder

Panoptic segmentation is a task of classifying each pixel in a natural scene image and finding the location,category,and mask for each instance in the image.At present,the panoptic segmentation,especially the end-to-end panoptic segmentation algorithm,has been studied less,but the research on semantic segmentation and instance segmentation has been well developed.Compared with the panoptic segmentation system composed of independent semantic segmentation network and instance segmentation network,the end-to-end panoptic segmentation network can make the semantic segmentation and the instance segmentation mutually promote,and has higher precision and faster speed.In order to analyze the complex natural scene images quickly and accurately,this paper firstly improved the semantic segmentation algorithm for the features of the stuff class in the panoptic segmentation,and proposed an end-to-end panoptic segmentation model based on the improved semantic segmentation algorithm.The main contents are as follows:Firstly,based on the DeepLab v3+,a semantic segmentation model that can accurately distinguish the small and fine categories in the stuff class is proposed.In order to fully express the features of different categories,this paper provided a backbone network based on the attention-based residual module.The backbone network combining with the ASPP module to form the encoder,which is used to extract features.The decoded features in the decoder are rich in abstract and detailed information.This paper proposed a scene information coding module,which enhances the judgment ability of the network for different categories to appear in the current scene through the information of the scene category,and achieves more accurate results.Experiments show the effectiveness of the proposed method,achieving an average accuracy of 53.4%mIoU and 74.56%on COCO 2018 Panoptic.Secondly,this paper proposed an end-to-end panoptic segmentation model based on encoder-decoder.The training of the end-to-end model can make the semantic segmentation and instance segmentation branches mutually promote each other.In this paper,a fast feature scale transformation module is proposed in order to improve the feature resolution without additional computational complexity.The semantic segmentation and the instance segmentation branch share features,and the outputs of the two branches are merged in the panoptic segmentation module.The prediction of the instances which block each other in the instance segmentation is adjusted by the semantic segmentation result.An unknown category is defined to reject the pixel of the undetermined category,reducing the influence of one pixel prediction error on the segmentation result.Finally output the panoptic segmentation of the natural scene image.Finally,the proposed end-to-end panoptic segmentation model is evaluated,achieving 40.1%PQ,77.8%SQ and 48.9%RQ on the COCO 2018 Panoptic dataset,and 59.3%PQ,79.4%SQ and 72.2%RQ on the Cityscapes dataset.