Research On RGB-D Semantic Map With Attentional Mechanism

The 3D maps obtained by traditional simultaneous localization and map construction(Simultaneous Localization And Mapping,SLAM)techniques have no semantic information and can only satisfy the localization needs to a certain extent.In order to further improve the intelligence of the robot,it is necessary to obtain a more conducive environment map for interaction.Meanwhile,aiming at the existing problems of most visual SLAM algorithms,this paper proposes a semantically based SLAM system to construct the semantic map of the environment,which can help robots to achieve higher level of environment perception and intelligent interaction tasks in complex indoor scenes.In order to complete the updating and correction of the map,and to improve the usefulness of the map,the obtained semantic map is converted to storage mode.The main research includes:(1)To address the problem that RGB images in indoor scenes are greatly affected by uneven illumination,a semantic segmentation method that fuses depth information is introduced to better distinguish semantic categories of indoor environments by introducing depth data that are robust to illumination changes.On this basis,the RGB image and depth image characteristic information for effective fusion,this paper proposes a fusion of semantic segmentation method is used to optimize channel characteristics,the introduction of attention mechanism is used to extract important features,and then to do optimization,RGB-D features step by step in the sampling process and effective integration,to capture the abundant characteristic information,improve the image semantic segmentation accuracy.Experimental results on indoor data sets NYUD V2 and SUN-RGBD show that the semantic segmentation network proposed in this chapter can be used in indoor environment.(2)Aiming at the needs of mobile robot autonomous localization and environment mapping,visual SLAM system is studied and designed in this paper.Due to the sparse information of the map point cloud constructed by traditional ORB-SLAM2,location information will be lost.Therefore,based on ORB-SLAM2,this paper firstly extracted the features of the input key frame images,and then converted them into the form of point cloud.Finally,the point cloud sequences were fused and spliced to obtain the 3D dense point cloud map of the global scene.Experiments verify that the algorithm can reduce the mismatching phenomenon caused by invalid information to some extent and improve the accuracy of graph construction.(3)By learning the image semantic segmentation model and analyzing the algorithm flow of visual SLAM,a semantic annotation environment map construction method was proposed.Firstly,the semantic information of objects in 2D images is obtained through semantic segmentation network,then the pixel association between adjacent images is established by visual SLAM algorithm,and the 2D semantic information is updated and fused into 3D environment map by Bayesian updating method.Finally,the semantic map is saved in the form of point cloud map.(4)In order to further reduce the storage space of the map,a semantic octree map is constructed by replacing point clouds with voxels,so that mobile robots can better complete obstacle avoidance and navigation tasks.In summary,the whole process first carries out semantic annotation for RGB-D images in the scene,then uses SLAM technology based on feature point method to complete 3D dense map construction of indoor scene,and finally integrates semantic information into 3D map to obtain3 D dense map containing semantic information.Experimental results show that the proposed semantic segmentation algorithm and semantic map construction method both achieve good experimental results,which lays a foundation for mobile robots to perform more complex high-level tasks in the future.