| With the rapid development of combat simulation technology in recent years,the command training based on simulation technology can effectively improve training effects and organizational efficiencies for personnel training.Building accurate Computer Generated Forces(CGF)behavioral models has an important impact on improving performances of the unit-level combat simulation system.As the basic content of CGF spatial cognitive behavior modeling,spatial exploration plays a supporting role in realizing a variety of high-level intelligent behaviors including path planning,tactical decision,intention recognition,etc.However,it is difficult for current approaches to meet the practical requirements of building combat simulation systems,mainly because(1)the existing rule-based spatial exploration methods do not consider the movement behavior of CGF,which is relatively mechanical in the process of spatial exploration;(2)most of the methods based on Machine Learning are end-to-end control,and it is difficult to train and complete exploration tasks in a short time.Moreover,irrational behaviors such as repeated oscillations and unsmooth movement often occur in the process of CGF movement.To tackle these problems,improving the cognitive rationality of CGF spatial cognitive behavior modeling and ensuring the operating efficiency of the systems,this paper mainly includes the following contributions and innovations:(1)A hierarchical framework of spatial exploration behavior modeling.In view of the shortcomings of the rule-based and machine-learning-based exploration methods,this paper creatively proposes a hierarchical spatial exploration framework which is divided into two layers.The high-level exploration strategy adopts the deep reinforcement learning method.By integrating the historical track,current observation and current position of CGF,it is endowed with spatio-temporal reasoning ability to determine the target points to be explored,so as to make the motion mode of CGF more in line with human behavior.The low-level movement strategy adopts the rule-based path planning method,which drives CGF to move from the current position to the target point according to the current known information,reducing the training difficulty of deep reinforcement learning,and standardizing the specific movement behavior of CGF.(2)A method of exploring static spaces based on intrinsic motivation is proposed.This paper proposed an autonomous spatial exploration method for static environment,IRHE(intrinsic rewards based hierarchical exploration).In the high-level exploration strategy,to solve the problem of sparse reward in deep reinforcement learning,based on the concept of intrinsic motivation in cognitive psychology,this paper designs an effective intrinsic reward function to accelerate the convergence of exploration strategy.In the low-level movement strategy,this paper designs an optimistic A~*algorithm,which can plan global path in partial observable environments,which effectively standardizes the mobility behavior of CGF.The simulation results show that compared with the existing methods,the proposed method covers almost all areas of the map,and the motion mode of CGF is more in line with human behavior.(3)A method of exploring dynamic spaces based on a hybrid algorithm which contents the optimistic A~* and self-adoptive finite-time velocity obstacle is proposed.To solve the problem of spatial exploration in dynamic spaces,based on IRHE,a hierarchical local mobility architecture is proposed.By combining the high-level path planning strategy with the low-level motion control strategy,the function of CGF real-time obstacle avoidance is realized.On this basis,this paper designs an optimal velocity evaluation function to select the optimal velocity in the current state,so as to realize the efficient exploration in the dynamic spaces.Simulation results show that this method can effectively explore the two-dimensional dynamic spaces on the basis of ensuring the safe movement of CGF.The mentioned works lay a good foundation for solving the important problems related to spatial exploration in the OODA decision loop of CGF and improves the fidelity and rationality of CGF behavior.To a certain extent,the proposed methods meet the realtime,authenticity and adaptive requirements of the unit-level combat simulation system,... |
