Multi-task Semantic Segmentation Method Based On Attention And Feature Fusion

The progress of deep learning has driven rapid advancements in the area of semantic segmentation in computer vision,which is crucial for the development of autonomous driving technology.As such,the need for highly accurate semantic segmentation models has grown tremendously in recent years.However,with the increasing demand,the deep model to solve semantic segmentation has reached a bottleneck period,and the accuracy of segmentation can only be obtained by continuously deepening the depth of the model.The reason is that the utilization of image information and the correlation between images have not been fully mined during the running of the model.This inspires us to efficiently use image information through feature fusion,and use multi-task learning methods to mine the complementary information of images between multiple tasks.Based on the verification of the effectiveness of the above methods,this thesis proposes two semantic segmentation methods based on multi-task learning from the three tasks of image semantic segmentation,depth estimation and surface normal estimation to simultaneously improve the accuracy of segmentation and estimation,and use fewer network parameters to alleviate the computational pressure of the model.The main work of this thesis is as follows:(1)This thesis proposes a multi-task semantic segmentation model based on attention and feature fusion.In this thesis,we address the problem of information loss by analyzing the high and low semantic information of images to alleviate the gradual loss of semantic information as the network depth increases.For parameter redundancy,this thesis introduces the feature mask generated by the attention mechanism in the design process to shield useless information to reduce parameter redundancy.By introducing the attention mechanism,it can not only ignore unnecessary information,but also exclude negative correlation information between tasks,so as to improve the training accuracy of all tasks and enhance the robustness of the model.Experiments on NYUv2 and Cityscapes datasets show that the proposed method not only improves the efficiency of semantic segmentation task,but also improves the efficiency of two estimation tasks,which affirms the efficacy of the suggested approach.(2)We propose a multi-task semantic segmentation model based on multi-scale feature fusion and distillation to fully mine the complementary information between related tasks.When building the multi-task learning framework,multi-scale image information is extracted in different depth network layers,and the loss of excessive spatial information in the down-sampling process is compensated by feature fusion and skip connection of different scales.The purpose of adding distillation is to coarsely process features by constructing intermediate auxiliary tasks,obtain high-quality features and reduce the amount of parameters in the middle stage of the model,and then improve the accuracy of its own task.The proposed method has three task-specific decoders to train segmentation and two estimation tasks.Experiments on NYUv2 and Cityscapes datasets show that the addition of multi-scale information improves the efficiency of semantic segmentation and two estimation tasks,the feasibility of the proposed method has been demonstrated.