Research On Reinforced Evolutionary Algorithm For Game Difficulty Control

In the field of game control,artificial intelligence(AI)technology has been widely applying,in which deep reinforcement learning(DRL)algorithms play an excellent role.The DRL algorithm is an algorithm that is based on environmental interaction and uses the rewards given by the environment to improve itself.It is also widely used to obtain a powerful agent.Especially with the improvement of hardware performance,the development of deep learning technology makes DRL Algorithms have been applied in real scenarios such as Atari games and Go and can obtain agents far exceeding humans.However,DRL has always faced some problems: sparse rewards,lack of efficient exploration,and sensitivity to parameters.These problems limit the application of algorithms in practical tasks.In the actual game control field,there are a lot of game difficulty control problems.According to existing research,most of the research work only focuses on training powerful agents,which are also called game AI,and there are relatively few researches on game AI difficulty control.Traditionally,the DRL algorithm controls the difficulty of the game by adding noise to the network of the AI.This method has poor stability.The evolutionary algorithm(EA)is a type of population-based algorithm.This type of algorithm does not need to perform a gradient search.More importantly,this type of algorithm can effectively avoid local optimal solutions.However,EA usually restricts by the scale of the problem,and it is difficult to optimize a large scale problem.To better solve such problems,this paper proposes Reinforced Evolutionary Algorithms(REAs),which are a combination of the DRL algorithm and the EA algorithm.REAs introduced the DRL algorithm under the framework of the evolutionary algorithm.The algorithm can effectively combine the advantages of the EA algorithm and the DRL algorithm.First,to improve the exploration ability of the DRL algorithm,the REA algorithm improves the exploration efficiency of the algorithm through evolutionary operations.Second,to make up for the lack of learning ability of the EA algorithm and unable to make full use of the information of the game environment,the algorithm combines the Actor-Critic algorithm in DRL to more efficiency for sample,so that the algorithm can converge faster.Third,to improve the generation of low-efficiency offspring waste CPU computing resources after evolutionary operations,and further improve the convergence speed of the algorithm,the algorithm proposes a classification strategy to classify the offspring of the population and then assigns different calculations to individuals of different categories Resources.Moreover,the algorithm also uses distributed parallel computing to update the neural network model of individual populations to improve the running speed of the algorithm.Finally,to solve the problem of insufficient population diversity in the REA algorithm,the paper models the upper and lower bounds of the optimization problem transforms the original single-objective optimization problem into a multi-objective optimization problem,and proposes a Reinforced Multi-objective Evolutionary Algorithm(RMOEA).Compared with the REA algorithm,the RMOEA algorithm mainly emphasizes the diversity of offspring individuals and uses the offspring with good diversity to achieve high-performance individuals.Experiments show that the REAs algorithm combines the advantages of the DRL algorithm and the EA algorithm to achieve better results in the field of game control,which shows that under certain conditions,compared with the DRL algorithm,evolutionary operations can make the policy more diverse.Compared with the EA algorithm,the REA algorithm introduces the gradient learning algorithm in DRL to improve the learning ability of the REA algorithm.Compared with the REA algorithm,the RMOEA algorithm can not only maintain the diversity of the population but also quickly obtain high-performance individuals.At the same time,in the field of difficulty control,the REAs algorithm can also generate game AI with different levels of difficulty more effectively.