Research On Cooperative Confrontation Of Multiple Agents Based On Deep Reinforcement Learning

With the rapid development of deep reinforcement learning,game AI has achieved good results in various game environments.There are many factors affecting the model ability of game AI,such as training core algorithm,feature processing method,neural network structure,reward function design and so on.This thesis studies the optimization of game AI model in multi-agent environment from three aspects: training core algorithm,feature processing method and neural network structure.Firstly,in terms of training core algorithms,the Factored Multi-Agent Centralised Policy Gradients(FACMAC)algorithm based on value decomposition is researched.This algorithm combines the advantages of value function decomposition algorithm and deep strategic gradient algorithm.The algorithm can be applied not only in continuous motion space,but also in discrete motion space,and the performance is better than other algorithms.However,like other depth strategic gradient algorithms,it will produce overestimate error of Q function.In order to solve this problem,In Thesis,a Double Factored Multi-Agent Centralised Policy Gradients(DFACMAC)algorithm based on double factored multi-agent centralised policy gradients is proposed to solve this problem through two Q functions.Thesis proves that this algorithm can solve the problem of Q function overestimation error from both theoretical and experimental points of view,and the actual performance is better than other algorithms.Then,from the perspective of feature processing method and neural network structure,the optimization method of AI model ability is proposed.Firstly,an improved algorithm scheme for feature communication during feature extraction is proposed.The addition of communication can enable agents to obtain information of friends and enemies in time,so as to make appropriate decisions and strengthen the collaborative efficiency between agents;Then the algorithm improvement scheme of customized neural network is proposed.The customized neural network based on specific environment and task can maximize the learning ability of the agent and improve the model confrontation ability.The final experimental results show that both of the two improvement schemes are helpful to improve the capability of the model.In conclusion,combining theoretical basis and practical experience,Thesis conducts in-depth research on theoretical optimization of deep reinforcement learning algorithm and actual performance of game AI model in multi-agent environment,and obtains better experimental results in different types of game environments.