Research On Indoor Multimodal Map Construction And Relocation Technology For Service Robots

Combined with the current trend of intelligence,people’s expectations for intelligent families are getting higher and higher.Service robots oriented to the home environment,which take service and security inspection as the main task,have become a research hotspot.This kind of robot can help the householder get items in the family environment,identify abnormal heat source,etc.For these tasks,the robot should be able to sense and understand the environment and carry out safety monitoring,and can run indoors for a long time.These functions require multi-modal information flow as input,so that the robot has multi-level environmental perception ability,so as to obtain the required measurement information,object information and temperature information,and use this information to perform tasks.Combined with RGB-D depth camera and thermal infrared imager,this paper mainly studies multimodal map construction and relocation algorithm research of indoor service robot.The research contents are as follows:The multi-sensor calibration and registration technology of mobile robot is studied to realize the multi-level information perception consistency of robot.This paper based on the indoor environment built figure structures,the robot platform,the relocation of the actual demand at the same time,carrying the multi-sensor vision system calibration RGB-D cameras and thermal infrared camera inside,in preparation for the 3 D environment modeling,and registration of the two cameras,make three kinds of image pixels corresponding synchronization in time and space,To provide visual information with high credibility for the robot’s perception system.In order to enable the robot to have multi-level environmental perception ability,this paper uses visible light information,depth information,object information and temperature information to input multi-modal information flow to enhance its ability to understand and explore the environment,so as to facilitate the subsequent robot to perform tasks.In this paper,the ORB-SLAM2 algorithm is used as the benchmark framework to gradually integrate RGB-D information,object information and temperature information,and gradually build three-dimensional point cloud map,threedimensional object-level point cloud map and multimodal map based on the region of interest.The multi-modal map based on the region of interest,including the type,size,spatial location of the object and the temperature of the region of interest,completes the robot’s perception and exploration of the unknown environment.In the home environment,the robot needs to work indoors for a long time,and it needs to find its global position and posture through relocation.For relocation at pixel level based on the traditional method for Angle of view for positioning failure occurs,illumination change is big problem,this paper USES the robot environment to explore the measures and the object information,extracting information of these two kinds of topology,graph matching method is put forward,the positioning problem into a graph matching problem,coarse location,then the target object point cloud segmentation,The specific pose of the robot is calculated by registration.Finally,based on the built robot experimental platform,this paper describes the 3D object-level point cloud map,the multi-modal map building form based on the region of interest,and the accuracy of 3D environment modeling respectively in terms of map construction.In terms of relocation,Experimental verification and discussion are carried out on the relocation accuracy of the proposed relocation algorithm and the classical algorithm as well as the relocation effect of the robot when the camera is restarted with a large Angle of view change.