| Image instance segmentation is to correctly detect objects in an image and semantically segment them at the pixel level,which is one of the pivotal technologies in understanding natural scenes.To achieve the instance segmentation task,the contour-based instance segmentation methods form the enclosed instance mask by predicting the set of instance contour vertexes.However,current contour-based instance segmentation methods generally suffer from contour decay and mask decay,which leads to a gap between the performance of the mask-based instance segmentation method.To alleviate these two decay problems,a unified refining structure is proposed in this thesis.Based on this refining structure,a high-performance contour-based instance segmentation framework is introduced,which achieves excellent instance segmentation performance on the COCO dataset.The main works of this thesis are as follows:(1)Taking the classic instance segmentation method Polar Mask as the basic framework,the Internal Center and Hard Sample Polar Centerness are proposed to solve the problem of excessive screening of candidate instances in Polar Mask,which constitutes the benchmark contour-based instance segmentation architecture.(2)To solve the problem of contour decay and mask decay in contour-based instance segmentation methods,the instance contour refining sub-module and enclosed region refining sub-module are proposed in this thesis.In the contour refining sub-module,the original contour vertexes are modeled as a satellite graph structure.By using the Star Transformer structure,the corresponding features of the original contour vertexes are updated dynamically.Based on the updated features of vertexes,an offset is predicted for each original contour vertex to achieve contour vertex refinement.In the enclosed region refining sub-module,an instance heatmap generation network is designed based on a prototype network,which can alleviate the mask decay problem.A unified Contour and Enclosed Region Refining Module(CORE~2)is designed by integrating these two sub-modules,which can improve the performance of each contour-based instance segmentation method.(3)According to the mask modeling characteristics of the contour-based instance segmentation methods and the idea of the Generalized Intersection over Union(GIo U)loss function,a Polar Generalized Intersection over Union(Polar GIo U)loss function is designed to achieve better contour vertexes regression performance.(4)To enhance the representation capability of instance features in contour-based instance segmentation methods,we design a feature interweaving network that can effectively fuse high-level semantic information and low-level spatial information in deep neural networks.Three sub-modules are designed in the proposed feature interweaving network,which can enhance the complementarity between different scales features,enhance the global image information in instance features,and fuse high-level and low-level features to generate instance features with stronger representation capability,respectively.The experimental results show that replacing the feature extraction module in CORE~2 with the proposed feature interweaving network can effectively improve the performance of instance segmentation.Finally,a better contour-based instance segmentation framework is proposed based on the instance segmentation benchmark network and the above four improvements.Experiments show that the proposed contour-based instance segmentation framework achieves a performance of 38.2AP on the COCO 2017 dataset,which is better than most current contour-based instance segmentation methods. |
