Research On Super-resolution Semantic Segmentation Method With Interpretability

With the development of computer vision in recent years,semantic segmentation technology that has different perception modes from humans and provides pixel level image understanding plays an important role in this field.With the rapid development of deep learning methods based on convolutional neural networks(CNN),semantic segmentation technology has achieved significant results in the fields of geological detection,autonomous driving,facial segmentation,clothing classification,and precision agriculture.With the continuous improvement of semantic segmentation accuracy,it is inevitable that deeper and more complex convolutional neural networks and more and more parameters are needed,which relatively require an increasing amount of computation.This poses a serious challenge to the real-time performance and device performance of semantic segmentation.Therefore,research on improving the accuracy of semantic segmentation without increasing additional computational complexity is of great significance.And increasingly complex convolutional networks also represent a more serious black box problem in the network.Deep convolutional models that cannot be well explained affect the confidence level of model work,and there is an urgent need for an effective method to explain the principle of model work.This article proposes a bilateral semantic segmentation method in a low resolution environment without additional computational complexity,as well as an interpretability method based on class activation maps that is no longer limited to classification tasks.The bilateral semantic segmentation method proposed in this article is suitable for low resolution environments.Based on super-resolution technology,an attention branch is designed to improve the accuracy of the entire model segmentation.The semantic segmentation model mainly consists of two branches,the first branch acts as a pipeline and can accommodate any currently popular semantic segmentation network based on encoding decoding structure.The second branch first improves the feature map through attention mechanism,and then uses super-resolution method to reconstruct highresolution images through the feature map,supplementing fine-grained information and structural information.The two branches finally converge in the fusion module,guiding the learning of the first branch through the second branch,improving the accuracy of semantic segmentation method learning,and improving operational efficiency by removing the second branch from the validation set.Among them,the feature maps obtained from the encoder part of the mainstream semantic segmentation method are shared in the second branch,without additional training time consumption.At the same time,the output image obtained by the decoder is subjected to one layer of upsampling to obtain an output of the same size as the original image,fully utilizing the true region of the original image.For the interpretability method in semantic segmentation tasks,this article designs an interpretability method mainly based on ablation,which can be applied to any convolutional layer in the semantic segmentation model.By resetting the convolutional parameters of a certain channel in any convolutional layer,and then using the current model parameters for semantic segmentation,the average intersection ratio of a certain category obtained under the current model parameters is compared with the average intersection ratio of the original model output without resetting the convolutional parameters.By analyzing the contribution of the current channel to the current segmentation task in a certain way,the class activation graph weight of the current channel is obtained,Afterwards,the activation map of the convolutional layer is linearly weighted to obtain the thermal map of the focus of the convolutional layer,which is different from the interpretability method under the target recognition method.At this point,a thermal map of the focus equivalent to the number of target classes can be obtained,allowing for detailed observation and analysis of the model’s focus on any target class in a certain convolutional layer.At the same time,interpretability methods such as confidence testing and integrity checks were evaluated for this method.