Research On Control Method For Manta Like Biomimetic Fish Based On Reinforcement Learning And CPG

The ocean contains abundant resources waiting for human exploration and development,and the demand for underwater robots is gradually increasing.In the long process of evolution,the swimming ability of fish in the ocean has been exceptionally developed,providing rich inspiration for human design of underwater robots.Compared to traditional underwater vehicles,robotic fish designed after fish have the advantages of high efficiency,low energy consumption,and high mobility.Manta rays have agile movements and wide pectoral fins that ensure stability during swimming.With the complexity of biomimetic fish tasks,higher requirements have been placed on their ability to independently complete tasks.Therefore,this thesis takes the manta ray bionic fish as the research object,designs the bottom motion controller based on CPG and the high-level controller based on reinforcement learning,and integrates the GPS/INS deep coupling navigation algorithm into the control method to realize the autonomous swimming control of the manta ray bionic fish under the water surface.The main research content of this article includes:(1)Taking the manta ray bionic fish as the research object,the bottom swimming controller based on CPG is designed as the "low-level central nervous system" of the manta ray bionic fish.By changing the parameters of the CPG model,different swimming modes can be achieved,including straight swimming,left turning,right turning,retreating,accelerating,floating up and diving down.(2)A high-level controller based on reinforcement learning is designed as the "advanced central nervous system" of the manta ray bionic fish.By obtaining the observation values obtained from the GPS/INS based positioning and navigation system,and outputting the control parameters of the bottom motion controller based on CPG,the fixed point swimming control task of the manta ray bionic fish is completed,that is,input the position of the target point,and control the bionic fish to swim to the targeted target point automatically.(3)Designed a 3D positioning and navigation technology based on GPS/INS deep coupling as a "sensory organ" to obtain environmental sensing data.The deep coupling combination method can still output speed and position data without GPS signal,and can regularly surface to calibrate INS data,which is suitable for the underwater work characteristics of biomimetic fish.Adding a depth gauge to the GPS/INS combination algorithm can achieve three-dimensional positioning,and after data processing,the observation values required by the reinforcement learning algorithm can be obtained.(4)The control method proposed in this article was simulated in a simulation environment to verify the effectiveness of the proposed method.The control parameters of the CPG controller were improved,and the optimal control method was obtained through experimental comparison.It is also compared with SIN-DDPG control method(sinusoidal controller as the bottom controller,reinforcement learning as the top controller)and CPGPID(CPG as the bottom motion controller,PID as the top motion controller)respectively.The results show that the control effect of the control method proposed in this thesis is far better than the two control methods compared.(5)A manta ray type bionic fish test prototype was designed,and the control strategy formed by the control method training proposed in this thesis was transplanted to the test prototype,and the control effect of the involved bottom motion controller based on CPG,underwater positioning and navigation based on GPS/INS,and fixed-point swimming control task of the high-level controller based on reinforcement learning were respectively verified in the artificial lake.The experiment proves that the control method proposed in this article can control the manta ray biomimetic fish prototype to achieve the control task we set.