Attention Mechanism And Deep Multiscale Fusion Network For Semantic Segmentation

Recent years have witnessed a spurt of progress in sensor technology,more 2D images,videos,and 3D data can be obtained.In the face of massive data,scene understanding is to capture the relevant knowledge or semantics in real life.Such demand is developing in full flourish.As the core problem in computer vision,scene understanding has attracted much attention.As a high-level visual task,semantic segmentation opens new horizons for scene understanding.Semantic segmentation has important application value in automatic driving,geological survey,face recognition,etc.Although the application of deep neural networks,computer vision and other related technologies has brought a breakthrough to the semantic segmentation.However,there are still some challenges,such as low precision of small object and multi-scale object segmentation,and blurred edge segmentation.Given these challenges,this thesis combines multi-scale fusion,attention mechanism,and multi-task learning to study the semantic segmentation of remote sensing images and 3D point clouds.The main contents of this thesis are as follows.1.The thesis proposed a boundary awareness and multi-scale fusion network for remote-sensing image semantic segmentation.Aiming at the semantic gap existing in the direct fusion of high-level and low-level features,that is,high-level and low-level features con-tain different semantic levels of information,which leads to the inaccurate edge location of ground objects.boundary awareness and multi-scale fusion network is proposed.The pro-posed network builds a boundary awareness module to obtain the edge information from the hierarchical features in a bottom-up manner.It emphasizes useful boundary information and eliminates noise information in low-level features under the guidance of high-level features,to obtain more complete and fine object boundaries.The effectiveness and feasibility of the network are evaluated by the overall accuracy and F₁score on the available remote sensing building datasets of Potsdam,Vaihingen and Massachusetts.2.The thesis proposed a multi-task attention learning network for remote-sensing image semantic segmentation.In view of the significant difference between the main body and the edge of objects in the computer vision characteristics,this work proposes a multi-task attention learning network.The network decouples the boundary information from the body information and proposes a boundary loss to supervise the boundary prediction task,to learn the boundary information pertinently.Moreover,the network uses an adaptive weighted multi-task learning to further balance the boundary prediction and semantic segmentation tasks.It makes the interaction between the two tasks better,and provides a supplement for each other to improve the segmentation effect and network generalization ability.The effec-tiveness and feasibility of the network are evaluated by the overall accuracy and F₁score on the available remote sensing building datasets of Potsdam,Vaihingen and Massachusetts.3.The thesis proposed a spatial-channel attention network for 3D point cloud semantic segmentation.In 3D point cloud segmentation,simple aggregation is often used to capture complex local relationships,which leads to insufficient extraction of point cloud context information.To solve this problem,a spatial channel attention network is proposed.The network constructs a spatial attention module to capture correlation between points within a local region,and builds a channel attention module to consider semantic information from a high level as a guide to weight the low-level feature information to obtain a finer segmen-tation.The validity and feasibility of the network are evaluated by the overall accuracy and mean intersection and union on the available point cloud datasets US3D and S3DIS.