Research On Real-Time Semantic SLAM System For Dynamic Environment Localization

With the rapid development of artificial intelligence technology,intelligent mobile robots with autonomous sensing ability have attracted more and more attention.As one of the key technologies for sensing,visual SLAM relies on camera sensors to obtain information about the surrounding environment and complete its localization.Based on the assumption of static scenes,most of the existing visual SLAM methods can run stably and robustly,while dynamic objects often exist in the real environment.While in dynamic scenes,original SLAM methods are largely affected by moving objects,leading to large errors in localization results.In order to solve the problem of localization and mapping in dynamic scenes,semantic SLAM combined with deep learning technology is widely used.In order to solve the problem of localization and mapping in dynamic scenes,semantic SLAM combining deep learning technology is widely used.Semantic SLAM takes advantage of the powerful image understanding ability of neural networks to provide priori contextual semantic information for SLAM,and combines the semantic information to process dynamic objects.Although current semantic SLAM methods can effectively improve the localization accuracy in dynamic scenes,they fail to further improve the real-time performance of the system.To solve this problem,this paper designs a real-time semantic SLAM system for dynamic scenes from the perspectives of model lightweight and dynamic probability optimization.The specific research contents are as follows:Firstly,most of the existing semantic SLAM methods use complex detection or segmentation model in the front end,which leads to slow running of SLAM system.To solve this problem,this paper proposes a semantic segmentation method for continuous images based on multi-level knowledge distillation,which can obtain a lightweight segmentation model suitable for continuous images.This method divided the knowledge of teacher model into high,middle and low levels for distillation.These levels of knowledge were respectively referred to as model predictions,multi-scale fusion feature maps and activations in intermediate feature layers,and distillation terms were designed based on multi-level knowledge.First,feature distillation based on low-level knowledge ensured the feature distribution between student model and teacher model could be as close as possible.Then,based on middle-level knowledge,spatial structure knowledge of images was transferred to the student model.Finally,high-level knowledge was used to encode the dependency between adjacent frames,then this implicit knowledge was transmitted to the student model.Moreover,additional semantic consistency loss effectively improved the inconsistent predictions between adjacent frames.Experimental results show that the distillation method can significantly improve the accuracy of the student model,and achieve a better balance between model accuracy and lightweight.Secondly,in order to further improve the running speed of semantic SLAM in dynamic scenes,this paper analyzed the factors affecting real-time performance of the system.Based on the framework of ORB-SLAM3,a real-time semantic SLAM method based on dynamic probabilistic optimization was proposed from the perspectives of neural network lightweight,keyframe processing strategy,probability propagation and update,and pose optimization.In terms of improve real-time performance,a segmentation model for continuous frames trained by multi-level knowledge distillation method was deployed in SLAM system,and the semantic segmentation module was set as a separate working thread to improve the parallelism of whole system.Only keyframes were used as the input of semantic segmentation module to avoid the time delay caused by processing each image.In addition,in terms of dynamic object processing,this paper proposed a static semantic keyframe screening method,which selects keyframes containing more static information and reduces the participation of dynamic objects.By assigning corresponding dynamic probability to extracted feature points,this paper carried out dynamic probability propagation and update for feature points on each frame by combining semantic segmentation results of keyframes and data matching algorithm,and finally used feature points whose dynamic probability was lower than threshold value for initial pose estimation.Finally,dynamic probability was combined with local BA.The corresponding weights of map points are obtained according to their dynamic probabilities,then weighted reprojection error are calculated by using these weights.Experimental results based on open datasets showed that this method can effectively process dynamic feature points and improve running speed of the system.In addition,based on the embedded AI development board Jetson TX2,the method was verified in the actual dynamic scene,and results showed that the method in this paper can run smoothly on the embedded platform.