Tensegrity Robot Locomotion Control Via Reinforcement Learning

Tensegrity is a stable self balancing system composed of a group of discrete compression members and continuous tension members.The tensegrity structure initially emerged in the field of sculpture,but later it was adopted in many large-span structural projects because of its novel shape,unique conception,light weight and certain stiffness.In recent years,because of its high strength to mass ratio,dispersible internal force and low cost,it has become an active field in robot research.Spherical tensegrity robot is used as space exploration robot.However,due to its unconventional structure and high coupling dynamics,the efficient motion control of tensegrity is still a difficult problem.It is difficult to achieve effective motion control using traditional control algorithm.Deep reinforcement learning algorithm has been used in many robot tasks because of its strong perception and decision-making ability.However,it usually needs to collect a large number of samples,which limits its application.The model-based algorithm can learn with fewer samples,but it has suboptimal results due to the accumulation of model errors.In this paper,the spherical tensegrity robot is taken as the main research object.Firstly,the dynamic model of the robot is established,and a hybrid algorithm of model-free reinforcement learning and model-based reinforcement learning is proposed to realize the efficient motion control of the tensegrity robot.Finally,the artificial potential field method is combined to realize the obstacle avoidance control under different obstacles.The specific research work is as follows: firstly,based on the position coordinate finite element method,the rod element and the cable element in the tensegrity structure are defined,the generalized force,the tangent stiffness matrix and the tangent damping matrix are derived,the mass matrix is obtained according to the kinetic energy expression,and then the dynamic equation of the system is obtained according to the Lagrange motion equation.Then,the numerical methods of nonlinear differential equations are introduced.Newmark method is used to solve the dynamic equation.In the numerical example,the dynamic relaxation method is used to complete the form finding analysis of the tensegrity robot.The simulation results are compared with those of commercial physics engine MUJOCO.Then,an intensive reward function is designed for the rolling control problem of tensegrity robot,and an improved method based on DDPG algorithm is proposed.The specific time limit is to establish the neural network dynamic model through the random sampling data,and use the model predictive control to get the preliminary control effect.The controlled trajectory is used to initialize the parameters and memory of DDPG.By training the parameters of DDPG algorithm,a high-performance control strategy is obtained.Experiments show that the sampling efficiency of the hybrid algorithm is much better than DDPG algorithm.The control effect of different ground conditions proves the efficient control ability of the algorithm.Finally,an obstacle avoidance algorithm combining artificial potential field method with DDPG is proposed.The control results of artificial potential field method using reinforcement learning algorithm with different exploration steps for tensegrity robot are analyzed and compared,and the obstacle avoidance strategies suitable for different obstacles are obtained.