Semantic SLAM Research In Dynamic Scenes Based On Deep Learning

With the wide application of robots in various fields in recent years,traditional SLAM algorithm based on geometry can no longer meet the demand of people who want robots to accomplish more complex tasks,and it is a hot direction for robots to realize semantic information perception of surroundings while autonomously localizing in unknown complex environments.Improving robots’ ability to perceive and understand semantic information about surroundings is of great significant for robots to accomplish more advanced and complex tasks and to enhance autonomy and robustness of exploring environment.In this paper,in dynamic indoor scenes,enables robots to perform semantic perception of objects in environment and construct semantic maps while estimating their own motion and posture.Specifically,the following aspects are included.Firstly,in order to improve robustness of visual SLAM algorithm in dynamic scenes and reduce interference of dynamic objects on performance of SLAM system,a dynamic feature points rejection method based on target detection was designed in this paper.By adding this method as a new thread to original ORB-SLAM2 algorithm,the impact of dynamic objects on system performance can be effectively reduced.In target detection module,a lightweight target detection algorithm was designed based on YOLOv3 in order to enable the algorithm to run in real time on mobile,first the original YOLOv3 backbone feature extraction network was replaced by the lightweight network Mobile Netv3,then CBAM attention mechanism was introduced on the three effective feature layers extracted by feature extraction network to make network to pay attention to target adaptively,and the loss function was also improved.Experimental results show that the improved lightweight detection algorithm achieves an m AP of 91.92% on publicly available dataset VOC07+12,with a detection time of 0.014 s for a single image,compared with original YOLOv3 algorithm,m AP is increased by 3.31% and detection speed is increased by 2.64 times.Both detection accuracy and detection speed meet the requirements of semantic SLAM system.After potential dynamic objects are detected by target detection module,detection and rejection of dynamic feature points are completed in combination with the improved optical flow method.Experimental results show that the improved ORBSLAM2 algorithm proposed in this paper is more robust in dynamic scenes and has substantially improved performance.Secondly,in order to obtain semantic information about each object in environment as well as to segment target,a lightweight semantic segmentation algorithm was designed by combining fully connected conditional random fields.The algorithm was based on the current excellent semantic segmentation algorithm Deeplabv3+,similarly,to be able to deploy on mobile,the backbone feature extraction network was replaced with a lighter one.At the same time,the empty convolution part of enhanced feature extraction network was decomposed into an asymmetric convolution structure to reduce calculation volume and improve model running speed.For the problem that segmentation accuracy decreased after network model was reduced,the algorithm combined fully connected conditional random fields to correlate global information of image to improve accuracy of algorithm.Experimental results show that the improved segmentation algorithm sacrificed 3.61 percentage points of m Io U in exchange for real-time performance on mobile side.Segmentation accuracy and operation speed meet the requirements of subsequent semantic map building.Finally,in order to incorporate semantic information in established 3D maps,the overall architecture of 3D semantic map construction was designed.A three-thread structure with improved ORB-SLAM2 thread,segmentation thread and semantic map building thread was adopted to ensure operational efficiency of system.The map building part built a library of objects based on the semantic information obtained by semantic segmentation algorithm and segmented results and updated them in time when system was running to provide semantic information for 3D semantic map annotation.Octree map was used as map storage form,and object semantic information was fused into 3D map by combining camera pose obtained by improved ORB-SLAM2 to complete the construction of 3D semantic map.Experimental results show that the semantic SLAM system proposed in this paper possesses feasibility and accuracy.