Position Control Of Hyper-Redundant Continuum Robot Based On Soft Actor-Critic Algorithm

Hyper-redundant continuum robots have excellent intervention capability in complex constrained spaces due to their flexible multiple-degrees of freedom.These robots are usually driven by cables,and after a long period of operation,the theoretical model is prone to deviate from the actual working conditions due to factors such as fabrication,wear and tear,and cable deformation,which in turn leads to a decrease in control accuracy.As a result,the controller design based on a priori knowledge often fails to maintain the control performance of its initial design.To solve this problem,this paper investigates the end position control of a continuum robot based on reinforcement learning algorithm,and the main works are as follows.(1)The theoretical basis of flexible actuation evaluation is established,and the advantages and disadvantages of two types of methods,model-free and modeled,are explored through research,and optimization methods such as iteration of pairwise ratio functions and strategy gradients are analyzed.Based on this,deep reinforcement learning and maximum entropy theory are discussed.(2)Reinforcement learning is modeled for the super-redundancy continuum end position control process,and the network structure is designed and the training method is explored based on the Soft Actor-Critic(SAC)algorithm for the problem of high dimensional space of continuum state input and action output,which makes training difficult to converge directly.In the sparse reward problem,a reward function consisting of end position,behavior guidance and action penalty is introduced,and the reward function is dynamically adjusted according to the training process to guide the convergence of the accelerated network.For the case of multimode distribution of optimal Q values due to multiple degrees of freedom of the continuum,the SAC action entropy design is combined with the introduction of exploration noise,which makes the action restricted randomization.A continuum simulation environment is established to verify the control feasibility of the SAC algorithm,and the analysis of the algorithm control performance is carried out.(3)To address the problem of low data utilization of the SAC algorithm,the bottom integrated state transfer neural network model is introduced,combined with model predictive control and cross-entropy methods for the design of the inference and decision part,and the model-based trajectory sampling method(MBTS)is established,and the feasibility of the algorithm is verified in the simulation.In the state transfer model,we compare the control effect difference between mathematical model and network model,explore the influence of hyperparameters on the control effect,analyze the target point arrival success rate in different control regions,and verify the stability of the inference decision layer.By adding error bias,the adaptability of the network model and the mathematical model is compared for the case that the actual model is changed due to wear and tear,deformation and other factors.(4)The algorithm verification experiments are conducted on a continuum solid robot based on the model trajectory sampling method.An experimental platform consisting of hardware such as a motion capture system is built,and a TCP communication mechanism is established.In terms of control algorithm application,circular motion trajectory planning and physical verification are completed using mathematical model and network model,respectively;complex trajectory motion planning is completed based on MBTS through a small number of training iterations,and physical verification is conducted on the continuum robot,and control accuracy within 1 cm is achieved in all three directions of XYZ.