Research On Motion Control Method Of Quadruped Robot Based On Deep Reinforcement Learning

With the development of computer technology and advancements in artificial intelligence research,robots have gradually emerged in the public eye.Legged robots,due to their outstanding mobility,have attracted widespread attention from researchers,making their motion control a focal point in the field of robotics.Research on motion control is a key focus in the field of robotics.The discrete footholds of legged robots give them greater adaptability in complex environments,enabling them to replace humans in hazardous areas.Their excellent motion characteristics have opened up significant prospects for development in areas such as rescue operations,military applications,and measurements.Quadruped robots have complex structures,and traditional motion control methods require the analysis,modeling,and parameter tuning of forward and inverse kinematics.These methods cannot achieve optimal control over terrains and lack adaptability.In contrast,motion control methods based on deep reinforcement learning enable autonomous learning of control strategies without the need for model establishment.With the continuous development of deep learning,motion control methods based on deep reinforcement learning possess both the powerful perception capabilities of deep learning and the autonomous decision-making abilities of reinforcement learning.This significantly enhances the motion performance and terrain adaptability of quadruped robots.The specific contents of the thesis are as follows:(1)Establishment of quadruped robot model and kinematic analysis.Firstly,the physical model of the quadruped robot was reproduced in a 1:1 scale in a simulated environment based on the robot’s physical parameters.Next,the kinematic model of a single leg of the quadruped robot was analyzed,and the forward and inverse kinematics as well as the Jacobian matrix were derived.Finally,various gaits of the quadruped robot and their characteristic descriptions were summarized.(2)Research on motion control methods for quadruped robots based on a virtual model.Firstly,the trajectory planning for the foot end of the quadruped robot was designed.An adaptive adjustment strategy was devised specifically for continuous uneven terrains to determine the gait timing.Next,the control methods for the swing and stance phases of the quadruped robot were analyzed.Additionally,the steering functionality of the quadruped robot was incorporated into the design.Finally,by utilizing the Jacobian matrix,virtual forces were transformed into joint torques,enabling the quadruped robot to achieve omni-directional trotting.(3)Research on motion control methods based on Deep Deterministic Policy Gradient(DDPG)deep reinforcement learning.In order to address the low efficiency of training in the early stages of reinforcement learning algorithms,a reinforcement learning controller was designed.This included the design of the state space,action space,termination conditions,as well as the deep neural network structure and parameters.From the perspective of the quadruped robot’s task objectives,the stability of the body during motion,and the coordination of rhythmic movements among its legs,a reward function was designed.The communication process between the reinforcement learning algorithm and the environment interaction was described.Autonomous walking of the quadruped robot was achieved on flat terrain through the application of this reinforcement learning approach.(4)A motion control method for quadruped robots based on prior knowledge in deep reinforcement learning was proposed.The method utilized the network output of Virtual Model Control(VMC)as prior knowledge to guide the learning process of Deep Deterministic Policy Gradient(DDPG).The network output of VMC served as the open-loop component,while deep reinforcement learning acted as the closed-loop adjustment component.This approach ensured stability during the motion process of the quadruped robot while accelerating the training speed of deep reinforcement learning.Finally,the effectiveness of the algorithm was verified through experiments involving ramps and traversing gaps.