| Adversarial intelligent game is the core of the future war.With the development of artificial intelligence theory and technology step by step to the military field penetration,machine game becomes more and more important in the forms of intelligent game.Currently the world’s military powers are stepping up research on how to enable auxiliary machine intelligence for human commanders to solve the problem of the game confrontation on the battlefield.Namely,they’re interested in the transformation of machine game theory and technology to military application.The research of machine game can be divided into two types according to whether the players have private information or not,that is,complete information game and incomplete information game.In the incomplete information game,due to the private information of each player,it is hard to know the true state of game,which leads to the players stuck with information asymmetry and incomplete.Therefore,the study of incomplete information game is generally considered to be more challenging,involving many key problems that need to be solved.For example,the opponent exploitation,in which participants need to explore the available information to provide more sufficient prior conditions for their own strategies.This plays a crucial role in incomplete information games.Although it is impossible to infer the state information and corresponding decision-making models of other players at each stage of the game,it is proved that most strategies are not perfect and available.Under this assumption,theoretically there is always an optimal strategy to maximize the benefits of the opponent,which is the core starting point of the study of opponent exploitation methods.The research of this thesis focuses on the opponent exploitation methods in the incomplete information game,and devotes itself to developing an intelligent algorithm framework that can effectively as well as efficiently learn the counter strategies of different opponents in the game environment,providing a new idea for solving the incomplete information game.The main work and contributions are as follows:(1)A novel opponent exploitation method based on neuroevolution is proposed.Neuroevolution is a kind of cross concept combining artificial neural network and evolutionary learning algorithm.It is an important branch of artificial intelligence and machine learning field,which is suitable for parameter optimization of complex neural network.This thesis focuses on opponent exploitation in the incomplete information game of Texas hold’em poker.Firstly,the decision model of Texas hold’em poker agent based on Long Short-term Memory neural network(LSTM)is built.Then we use the “algorithm population” training framework of neurovolutionary method to iteratively learn the counter strategies under different opponent strategies,and prove the effectiveness and feasibility of this method through specific experiments.The experimental results show that in the face of four typical styles of opponent strategies,the total return of opponent using strategy in this chapter can be increased by about 13.8 times compared with the benchmark strategy(2)An improved opponent exploitation method combining reinforcement learning with neuroevolution is proposed.The neuroevolutionary method on opponent exploitation is one of the important ideas to solve the incomplete information game problem.Its essential idea is to use the algorithm “population” form of redundancy and diversity to meet the challenges of the complexity the solving problem But in a certain extent,it exists drawbacks of low utilization rate of samples and training time consuming.In the evolution process of “population” constantly interacting with opponent strategy,a large amount of interactive data have not been preserved and fully utilized.To solve this problem,this thesis proposes the method of off-policy reinforcement learning to improve the efficiency of the opponent exploitation algorithm based solely on neuroevolution.(3)Aiming at the opponent exploitation method designed for online Texas hold’em poker game platform,a synthetic approach based on opponent strategy recognition and exploitation is proposed.Previous opponent modeling and exploitation methods usually try to find the opponent behavior rule as a part of its strategy planning.Such idea inevitably requires complete historical data and experience knowledge.However,the incomplete information game such as Texas hold’em poker faces challenges of high dimensionality and complexity of game space.Extracting the available decision rule information from opponent behavior by traditional methods is not only time-consuming and laborious,but also difficult to ensure its reliability.In this thesis,we focus on the strategy level,using bayesian classifier to group opponents’ possible decision models into several benchmark style type.And based on the proposed opponent exploitation method combining artificial neural network and evolutionary learning algorithm,counter strategies for each strategy type can be trained offline.During the online competition scenarios,our agent has the ability to dynamically switch its strategy depending on the opponent’s strategy type,which can lead to maximum rewards in the game confrontations against multiple opponents. |
