Research And Implementation Of Intelligent Confrontation In War Game Based On Reinforcement Learning

With the development of modern computers and the advancement of artificial intelligence,military information technology,command systems,and decision-making systems are gradually becoming intelligent.As an important way to calculate strategic trend,wargames also need to be combined with computer technology and artificial intelligence in order to stay at an advantage in the ever-changing international situation.Reinforcement learning,as one of the most popular parts of artificial intelligence,is very suitable for use in the wargame environment.Researchers usually use images as the state space for reinforcement learning training,and discard other useful information contained in the wargaming environment,such as the location and movement direction of each unit.If this information can be used effectively,the wargaming system can be made more intelligent.This article combines the influence map,another decision-making method in artificial intelligence,with the original image state space of reinforcement learning,and designs and implements artificial intelligence decision assistance based on deep reinforcement learning.Starting from the level of intelligent situation description,it can describe the situation in real time,make quick response,and provide intelligent support for the decision-making system.The main work of this paper is as follows:(1)Aiming at the research of intelligent situation description,this article takes the custom-implemented small intelligent confrontation game Ms.Pac-Man as an example,calculates the influence map based on the internal state of the game and other information,and designs the influence map superimposed image as state space.In order to prove the validity of the state space proposed in this paper,experiments are carried out with the image state space as a comparison.Experiments have proved that compared with the traditional continuous image state space,this improvement improves the training speed by 59%,reduces the memory usage by 30%,and reduces the memory usage by 50%.The final score of the agent is increased by 10%.(2)The sparse reward problem in reinforcement learning is also the current technical difficulty.For the sparse reward problem in reinforcement learning,this paper uses an influence map to return the intrinsic reward to the agent when there is no external environmental reward.Experiments show that the final score of the agent is increased by about 10%compared to when there is no intrinsic reward.(3)For developers who try to combine artificial intelligence and wargaming,this article designs and implements an online interactive algorithm platform for reinforcement learning,which can add reinforcement learning support to the environment without changing the original wargaming environment.The users can view the progress of training in real time,which can be used as a reference for the customized wargame environment.