Research On Control Algorithm Of Bicycle Robot Based On Inverse Reinforcement Learning

Bicycle robot is an important research topic in the field of mobile robot.Bicycle robot has the advantages of simple structure,low energy consumption,and high passability in winding and narrow road conditions.Therefore,the design of a stable and reliable bicycle robot balance controller has high practical value and practical significance.This paper presents a study on the design of a balance controller for bicycle robots that can take full advantage of the prior knowledge contained in the mechanistic model and improve the system’s environmental adaptability through a data-driven method.Firstly,from the model-driven perspective,based on the semi-empirical dynamics modeling algorithm,this paper optimizes the feature extraction and feature selection methods and proposes a semi-empirical dynamics modeling algorithm for bicycle robots based on feature selection and RHONN.This dynamic modeling algorithm makes full use of the prior knowledge of the target system and can learn the dynamic characteristics of the unmodeled and uncertain system in the mechanism model by the data-driven method.The dynamics model of the bicycle robot is established based on the dynamics modeling algorithm,and the balance controller of the bicycle robot was designed based on the inverse optimal control algorithm.The balance control of the bicycle robot is realized preliminarily.The model-driven inverse optimal control algorithm and the datadriven inverse reinforcement learning algorithm are both algorithmic frameworks for solving the inverse optimal control problem.The inverse optimal control algorithm makes full use of the prior knowledge contained in the dynamic model and has higher design efficiency,but requires manual parameter tuning.The inverse reinforcement learning algorithm learns the reward function and the corresponding control strategy in a data-driven manner,which does not require a prior knowledge of the system and has a strong environmental adaptation capability.However,the lack of prior knowledge also means that random control strategies and random reward functions are often used at the beginning of training,which greatly reduces the training efficiency.To address the above problems,this paper uses the inverse optimal control algorithm to improve the training efficiency of the inverse reinforcement learning algorithm from two aspects.On the one hand,since the cost function of the inverse optimal control algorithm and the reward function of the inverse reinforcement learning algorithm are two formulations of the same problem in different perspectives.Therefore,this paper designs the structure of the reward function in inverse reinforcement learning algorithm based on the cost function V(x(k))of the inverse optimal control algorithm,and uses the parameter matrix P to initialize the parameter vector θr of the reward function.After the initialization,the inverse reinforcement learning algorithm is applied to further iterate the parameter vector θr in a data-driven way.On the other hand,it is necessary to combine a forward reinforcement learning algorithm to solve the optimal control strategy corresponding to the current reward function for a reverse reinforcement learning algorithm.To improve the training efficiency of the forward reinforcement learning algorithm,this paper uses the inverse optimal controller designed based on the dynamics model as a reference to initialize parameters of the reinforcement learning controller and reduce the state space range which the reinforcement learning algorithm needs to explore in the early stage of training.Based on the above approach,this paper constructs a bicycle robot balancing controller with good dynamic performance and anti-interference capability.