Research On Visual-based Online-robust Environment Perception And Navigation Method

As we all know,mobile robots have been widely used in various industries in social production life,such as indoor service and hospital logistics transportation,etc.Long-term autonomous navigation capabilities are the basis and prerequisite for mobile robots to perform tasks safely and reliably.Environmental perception and path planning are key core technologies for mobile robot navigation.Among them,visual environment perception and path planning have attracted wide attention from researchers with its light and inexpensive visual sensor,rich amount of collected information,and highcost performance.However,most of the current visual environment perception and path planning methods rely on low-dimensional pixel-level landmarks and their constructed geometric metric maps.On the one hand,pixel-level landmarks are less robust to weak visual factors such as lighting changes and weak textures.On the other hand,geometric metric maps have large storage space requirements and high requirements for computing resources.Therefore,given the difficult problems of ultra-lightweight and strong robustness in most of the current visual environment perception and path planning methods,this thesis proposes an object-level visual environment perception and navigation framework based on high-dimensional object-level landmarks,and studies specific methods such as object-level environmental modeling,adaptive global path planning and active local path planning.The main research contents and contributions of this thesis are as follows:Firstly,to achieve lightweight and robust environmental perception,an environmental perception modeling method based on object-level landmarks is proposed.Inspired by human perceptual navigation,the object-level landmarks and their semantic properties are extracted by means of 2D object detection and deep information fusion,meanwhile,the graph structure is used to organize and store the object-level landmarks of the environment.To improve the accuracy of the environment model,a global location method based on scene semantic graph matching is studied,by using a multi-attribute scene semantic diagram descriptor based on random walking to describe the object-level landmarks,combined with the multi-constraint-based graph matching method,the accurate and robust global positioning in the environment modeling process is realized.To improve the efficiency of environmental perception under limited computing resources,a hierarchical memory management mechanism is studied.Experimental results show that the modeling rate of the environment on the embedded platform can reach 11 frames per second,and the model storage capacity is reduced by nearly 23 times compared with the grid model,which verifies the effectiveness of the proposed method.Secondly,to effectively improve the robustness and reliability of visual global path planning in complex indoor scenes,an adaptive global path planning method based on object-level landmarks guidance is proposed based on the object-level environment model constructed above.By using 3D object detection to directly obtain object-level landmarks,the semantic properties of landmarks and geometric spatial information are used to construct high-dimensional semantic geometric features,and the robustness of visual environment perception is further improved.To select the optimal global topological path,the heuristic graph search method based on object-level landmarks,and a global path segmentation strategy based on active visual perception and object guidance are studied,which reduces the global cumulative error in the navigation process.Meanwhile the trajectory generation and refinement are carried out through the Bernstein polynomial to achieve adaptive smooth trajectory generation.Experimental results show the method proposed above has significantly improved reliability and robustness.Finally,to improve the safety and reliability of visual local path planning for complex and large indoor scenes,an active local path planning method based on visual perception prediction is proposed.To improve the robot’s perception detection ability of the invisible area,a local environment modeling method based on timing consistency is studied,and the projection of 3D geometric features on the 2D plane based on the object-level landmarks realizes the effective description of the invisible local area.Besides,based on the extracted object-level landmarks mentioned above,a visual-based local spatial perception prediction method is studied,which realizes the effective description of spatial obstacles by converting the semantic geometric space information of object-level landmarks into two-dimensional pseudo-laser data,and improves the robot’s perception and ability of incompletely visible obstacles.To obtain the optimal local obstacle avoidance trajectory,a local trajectory optimization strategy based on visual is studied,and the dynamic selection of the optimal local trajectory is realized by optimizing the evaluation function based on the above visual perception information.Experimental results show that the method proposed above has safe and reliable spatial obstacle avoidance performance.