Research On Visual Servoing Of Mobile Robots Based On Simultaneous Localization And Mapping

Visual servoing of mobile robots aims to drive the robot to a desired pose or track a reference trajectory via real-time feedback images captured by the onboard camera.In recent years,it has been widely applied in practice,such as security inspection,intelligent warehousing,and autonomously parking,becoming one of the hottest research topics in the robotic field.However,the field of view(FOV)constraint of the camera,unknown depth of the monocular camera,and motion constraint of mobile robots limit applications of existing visual servoing approaches.Specifically,most visual servoing strategies depend on artificial landmarks for pose calculation.In case of large lateral displacement to the desired pose,large steering is required since the mobile robot cannot move laterally,while the reference landmarks get out of the FOV frequently.Moreover,once the reference landmarks are occluded by dynamic objects in environment or pedestrians,the visual servoing task will be failed due to the interruption of feedback signals.Additionally,unknown depth of the monocular camera,velocity and acceleration saturation constraints of the mobile robot,dependency on applicable scenarios are unfavorable for the performance and efficiency of the mobile robot visual servoing system in practical applications.For aforementioned challenges,a visual servoing framework based on simultaneous localization and mapping(SLAM)for mobile robots is proposed in this dissertation.On the basis of the framework,several visual servoing strategies have been proposed to tackle the above challenges as follows:(1)Visual servoing strategy based on SLAM for mobile robots without artificial landmarks.The dissertation proposes a visual servoing framework based on SLAM to relax the original FOV constraint of the visual servoing system by means of calculating the current pose of the mobile robot based on the map in SLAM module.Owing to connections between the desired image and the feature map,feedback signals for controllers can be always available even in tough cases that the target scene is occluded or gets out of the FOV of the camera,improving robustness of traditional visual servoing systems.Meanwhile,the unknown pose scale can be fixed by the map rather than artificial landmarks,which makes it applicable in scenes without any artificial landmarks.Experimental results demonstrate that the framework relaxes the FOV constraint.Since mobile robots with nonholonomic constraint cannot move laterally,the path adopted in this framework is more effective than the zigzag path satisfying the FOV constraint in case of large lateral distance to desired pose.(2)General visual servoing strategy for multiple scenes based on integrated interactive mechanism between visual servoing and visual SLAM.Based on the visual servoing framework using SLAM,this dissertation designs an integrated interactive mechanism for applications in multiple scenes.For the unknown environment,a heuristic map initialization approach is designed,which selects a more suitable map module by means of either homography matrix or fundamental matrix.Thus,the visual servoing strategy is applicable in both planar and non-planar scenes.Additionally,a two-step algorithm for desired pose calculation is proposed to associate visual and control modules in the servoing system,which helps to provide feedback signals for the controller in case of occlusions in the target scene.Moreover,in order to improve feasibility in dynamic scenes,velocities designed by the controller are adopted in the SLAM module for pose prediction.The predicted pose is utilized for rejecting dynamic map points,making it more robust in dynamic environments.Several experimental results validate that the visual servoing strategy based on the integrated interactive mechanism is applicable in planar,nonplanar and dynamic scenes.(3)Visual servoing strategy based on command filter backstepping controller.In order to decrease execution errors resulting from abruptly changed velocities,acceleration signals are designed and then integrated to obtain smooth velocity signals.Thus,errors of the predicted pose can be reduced.In the graph-based SLAM module,motion constraints among pose nodes are added to improve the accuracy of feedback pose estimation.Combining vision and motion information,the unknown scale of pose resulted from unknown depth of monocular camera can be identified so as to reduce visual servoing model complexity.In addition,the servoing controller based on command filters is proposed to simplify the computation process of virtual signals derivation.Stability of the closed-loop system is analyzed by Lyapunov-based techniques.Eventually,comparative simulation and experimental results verify high performance of the visual servoing strategy.(4)Path-planning-guided visual servoing strategy with a trade-off between vision localization accuracy and motion efficiency.On considering both velocity and acceleration saturation constraints,accumulated errors in the SLAM module,and low efficiency of the redundant zigzag path for satisfying the FOV constraint,the dissertation proposes a path-planning-guided visual servoing strategy to balance the visual accuracy and control efficiency,which satisfies both velocity and acceleration saturation constraints.Firstly,candidate goals with velocity commands meeting saturation constraints are listed by the dynamic window approach(DWA).On the one hand,the SLAM module predicts observations of map points of the candidate goals to guarantee visual accuracy.On the other hand,Dubins paths from candidate goals to the desired pose are evaluated for servoing efficiency.The utility of candidate goals is calculated with weighted score of both accuracy and efficiency.It is noticeable that the weights are determined by uncertainty of current pose to balance visual accuracy and servoing efficiency.Specifically,in case of current pose with large covariance,repeated map point observations help improve accuracy of pose.Otherwise,if the covariance of current pose is tiny,short path designed by path-planning approaches will contribute to high efficiency.In summary,the path-planning-guided visual servoing strategy satisfying both velocity and acceleration saturation constraints of mobile robots balances visual localization accuracy and control efficiency.Comparative simulation results verify the superior performance of the proposed path-planning-guided visual servoing strategy.