Anti-interference Motion Planning Of Planetary Manipulator Based On Deep Reinforcement Learning

With the successful development of China’s Chang ’e lunar exploration project and the successful landing of Tien Wen 1 on Mars,the manipulator for planetary surface operation is the main execution unit of the mission.It provides reliable support for the subsequent implementation of important tasks such as the construction of extraterrestrial bases and large-scale inspection of the planet’s surface.This kind of more complex,long period,unknown environment mission scenarios put forward higher requirements for the planetary surface manipulator’s autonomy and antiinterference ability.As an important support to realize autonomous operation of manipulator,motion planning technology has been widely studied in academia and industry.In recent years,thanks to the rapid development of online computing and model-free learning technology,the robot arm motion planning technology based on reinforcement learning has gradually become a research hotspot.Compared with the traditional planning method,it shows stronger anti-interference and adaptive ability.Based on the reinforcement learning motion planning of the single agent manipulator,this paper proposes a discrete processing method for the traditional single agent,and improves the robustness of the original single agent through the centralized reinforcement learning of the discrete multi-agent,so as to have the anti-interference ability to various kinds of interference.Firstly,the motion planning method of single agent manipulator based on reinforcement learning is studied comprehensively.Considering the high-dimensional continuity of the planning space of the manipulator motion planning task,a Soft-ActorCritic(SAC)algorithm is adopted to study and train the manipulator motion planning task.Aiming at the bottleneck that the higher the planning precision is,the more difficult the training is,this paper proposes a new way of planning by combining artificial potential field method and reinforcement learning based on the analysis of five learning methods.By designing a flexible switching mechanism,reinforcement learning is used for planning when the distance is far from the target,and artificial potential field is used for planning when the distance is less than the threshold,so as to improve the speed and accuracy of learning.Through the training and testing of different precision planning tasks in the simulation engine,the advantages of the proposed method in high precision planning tasks are verified.Finally,combined with the static and dynamic grasping tasks of the robot arm in the real scene,the feasibility of directly transferring the learned planning strategy network to the real robot arm for use was verified.Secondly,in view of the weak anti-interference ability of the neural motion planner,based on the research on the motion planning of the single-agent manipulator,a discrete reinforcement learning motion planning method of the "multi-agent" manipulator was presented.Based on the establishment of the joint diagram of the manipulator and the analysis of the correlation relationship,a new method of multi-agent decomposition for a single manipulator is proposed,and the framework of multi-agent motion planning combining the centralized multi-agent SAC reinforcement learning method and artificial potential field is given.Finally,the simulation engine for multi-agent reinforcement learning motion planning of training and testing,through in the process of motion planning respectively random disturbance,joint locking action and disturbance,and effective to verify the effectiveness of the proposed discretization of multi-agent reinforcement learning mechanical arm motion planning method is compared with the traditional single agent has stronger anti-interference ability.