| As an important branch of deep learning,the convolutional neural network(CNN)has been widely used with the rapid development of artificial intelligence.Image segmentation technology,as a hot and difficult task in computer vision(CV),plays an important role in the development of autonomous driving,intelligent transportation,medical imaging,and other fields.Thanks to the improvement of computing power and the creation of a large number of labeled datasets,the training convergence problem of CNN has been well dealt with,and CNN has also achieved state-of-the-art(SOTA)performance in multiple CV tasks.Restricted to labor costs and modern technology,the creation of pixel-level annotations is difficult in many applications.Thus,we need to take full advantage of the weak annotations for image segmentation.Meanwhile,as a black-box system,the credibility of the output by CNN is often doubted.Moreover,in many applications where it is inconvenient or difficult for experts to intervene,a fully supervised dataset is hard to be obtained.Meanwhile,fields that are sensitive to the credibility of decisions,such as medical diagnosis and autonomous driving,have higher requirements for the visualization of each network decision.Thus,optimizing CNN,giving a visual explanation for the decision of CNN,and making full use of the visual results to output a pixel-level result on an incompletely labeled dataset will be significant research.The main contents of this dissertation are listed as follows:Due to the lack of local normalization ability of the convolutional layer in CNN,the convolutional layer is easily affected by local abnormal features,e.g.,local illumination intensity variation and local irrelevant features in the input image.Aiming at the problem,this dissertation proposes a batch covariance layer(BCL)algorithm to normalize the local abnormal features in the image and decrease the output deviation of the convolutional layer.BCL adopts a normalization of the full-channel feature maps in kernel size,as a supplement to the existing normalization methods,which can effectively reduce the influence of local illumination intensity variation and local irrelevant features.By converting BCL into several convolution operations,the method uses the existing GPU acceleration algorithm.BCL improves the recognition accuracy of CNN in multi-label classification tasks with little increase in computational complexity,which helps to improve weakly supervised image segmentation.Several visualization methods show that the fully connected layer is considered to merge the features extracted by the convolutional layer.However,the positive and negative gradients in the fully connected layer play different roles in merging category-related features.Aiming at the problem,this dissertation proposes a gradient rectified parameters unit of the fully connected layer(GRU-FC)algorithm.GRU-FC rectifies the negative parameters that generate negative gradients of the fully connected layer in the backpropagation procedure.GRU-FC focuses on the features more relevant to the category,to obtain a better clustering effect and improve the classification accuracy.Meanwhile,this dissertation proposes a simplified version for the CNN with single fully connected layer.The convergence of this method is analyzed experimentally,and its improvement of precision is compared with the standard fully connected layers.The classification accuracy of CNN on multi-label classification tasks is improved,and GRU-FC helps improves the performance of visual explanations and weakly supervised image segmentation.Existing CNN visualization algorithms focus on the output feature maps of the last convolutional layer,the class activation map(CAM)obtained is relatively rough after a series of pooling operations,and the resolution of the visualization decreases seriously.Aiming at this problem,a method for high-resolution grad-based CAM(HRCAM)of shallow layers is proposed.This method compares the contribution of the output feature map and gradient map for visualization,eliminates the output feature map commonly used at present,and only selects the gradient map for visualization.HRCAM introduces the gradient rectified method for optimization.By introducing the anti-adversarial(Adv)method,the output value of the target category is increased after several iterations of optimizing the input image,and the visualization results by Adv-HRCAM can obtain a class activation map covering more category-related areas.Meanwhile,aiming at the problem that semantic segmentation often relies on pixel-level labels for fully supervised methods,but the labor consumption of pixel-level labels is huge,this dissertation introduces HRCAM for multi-label classification models to design a weakly supervised semantic segmentation(WSSS)method.In this paper,the class activation map obtained by HRCAM is regarded as the initialization seed.After optimization,the effect of WSSS has been significantly improved.Experiments prove that our method with image-level labels merely outperforms earlier fully-supervised models,achieving SOTA performance of WSSS in image-level labels.The weakly supervised instance segmentation(WSIS)problem needs to distinguish different individuals of the same category in one image.Besides,the multi-label classification network based on image-level labels can not distinguish instances of the same category in one image,which leads to the poor performance of WSIS.Aiming at this problem,a WSIS method incorporating bounding box(Bbox)labels is proposed.The methods of introducing Bbox in this dissertation contain two aspects: directly referencing Faster R-CNN to predict Bbox labels to optimize segmentation seeds;visualizing the Faster RCNN network and merging with the visualization results of Adv-HRCAM as segmentation seeds.Both methods have great improvements in WSIS tasks,and both can surpass the SOTA method.The visualization of Faster R-CNN can not only obtain the SOTA WSIS score but also intuitively demonstrate the instance segmentation ability of the Faster RCNN.This dissertation conducts research on weakly supervised image segmentation based on improved CNN visualization and proposes optimization methods in multiple aspects involved in this problem.This dissertation not only proposes a better multi-label classification model with optimized convolutional and fully connected layers but also introduces a higher-resolution visualization method for the improved model.The visual results are used as seeds,after optimization,the WSSS results can be improved.By introducing Bbox labels to optimize segmentation seeds,this dissertation achieves SOTA results on the harder WSIS task.In addition,this dissertation visualizes the Faster R-CNN model to verify its instance segmentation capabilities. |
