UUV Intelligent Countermeasure Decision Making Based On Deep Reinforcement Learning

Along with the development of the unmanned coupled vehicle technology,unmanned coupled device will be the future high technology counter force to be reckoned with in the naval battle field conditions,but fight is the key to success lies in not only the equipment itself,the more is how to comprehensive planning,effective use and accurate monitor of forces,make a series of command decision-making right,this also for the intelligent and autonomous ability had the higher request,the intelligent decision is especially important in the process of attack.Because of its good model adaptability and learnability,deep reinforcement learning can be applied to the torpedo attack decision-making process of UUV by learning and finding the optimal solution of the problem in the antagonistic environment.In this paper,the sparse reward problem and sequential decision problem of UUV torpedo attack are considered and improved reinforcement learning decision problem are studied.Mathematical modeling and simulation program design are carried out for the basic attributes of the research problem.The algorithm is simulated and verified in maneuvering target shooting and red-blue countermeasures tasks.It mainly includes the following aspects:(1)This paper firstly analyzes the countermeasures,defines the basic attributes of UUV in the countermeasures environment,conducts mathematical modeling for the torpedo attack process of UUV,and designs a simulation environment including sonar detection,maneuverability and torpedo guidance.Then the scene elements are designed in detail and the mathematical model is realized as the simulation system program.Finally,the input and output of the decision system are determined according to the countermeasure environment model,and the interaction process between the decision system and the countermeasure simulation environment is clarified.(2)For unmanned coupled with a torpedo attack against that exist in the decision making of intensive study to solve the problem of sparse reward by reusing the unsuccessful history method adds additional reward feedback to solve,and in unmanned coupled with maneuvering target of torpedo shooting tasks,to verify the effectiveness of the algorithm,the improved algorithm is analyzed and compared and the efficiency of the conventional algorithm,the results prove that improved algorithm has higher decision-making ability.(3)Considering the reinforcement learning algorithm using depth decisions cannot be used for a long time of history information,the depth of reinforcement learning method combined with both short-term and long-term memory network,in which implement maneuvering target shooting scenes and red and blue both sides fight scene,to improve the algorithm is verified by the results show that the historical statusinformation for depth of intensive study of decision-making has a positive impact.