Uncalibrated Visual Servo Control Of Manipulator Based On Improved Reinforcement Learning

Uncalibrated visual servo control of manipulator does not require to accurately calibrate camera parameters,it uses the image information acquired by the camera to control the motion of the manipulator in Cartesian space,which enhances the antijamming capability of manipulator to the control environment and improves the robustness of the control system.Uncalibrated visual servo design control law in the image space,using online estimated image Jacobian matrix to reflect the nonlinear relationship between image feature change and joints angle change of manipulator.However,since the accurate visual mapping model is not established,the tracking of the optimal motion trajectory for the manipulator in Cartesian space by uncalibrated visual servo is not ideal.In view of the problem,this paper introduces an improved reinforcement learning algorithm.Reinforcement learning has the advantages of model-free and self-learning,which enables the manipulator to approach the optimal motion trajectory by autonomous learning.For the problems that the standard reinforcement learning algorithm has slow convergence and low learning efficiency,this paper optimizes the action selection strategy of reinforcement learning based on prior knowledge,shortens the motion exploration time,and improves the learning efficiency of reinforcement learning.The main research work of this paper is as follows:(1)An uncalibrated visual servo control method for manipulator based on improved reinforcement learning is proposed.Firstly,reinforcement learning is improved based on prior knowledge.The experience information obtained by the interaction between the manipulator and the environment is taken as prior knowledge,using the prior knowledge to construct an action selection function,and combine it with ?-greedy algorithm to form an action selection strategy of reinforcement learning.This strategy is used to guide the action selection of reinforcement learning,reduce the number of times to explore the optimal learning action,and reduce the amount of learning tasks.Then,based on Kalman filtering algorithm,the image Jacobian matrix and the posture of the end of the manipulator in Cartesian space are estimated online.The improved reinforcement learning algorithm evaluates and adjusts the control strategy according to the posture,and then optimizes the motion trajectory of the manipulator in Cartesian space.(2)Under the MATLAB simulation platform,the simulation environment is build by using the robotic toolbox,and the uncalibrated visual servo simulation experiment of the manipulator based on the improved reinforcement learning is carried out.By comparing with the simulation experiments based on the standard reinforcement learning algorithm,it is verified that in the simulation environment,the manipulator controlled by the method proposed in this paper has better motion trajectory in Cartesian space.(3)Based on Denso Robot Company's six-degree-of-freedom industrial manipulator,RC7 M controller,CCD industrial camera and PC computer,a calibration-free visual servo experimental platform for the manipulator is built,and the calibration-free visual servo spatial positioning experiment of the manipulator is carried out by using the method proposed in this paper.Through comparative experiment,it is verified that the method proposed in this paper has better control effect in the physical platform as well.