Research On Intelligent Radar Anti-Jamming Game Strategy In Time-Frequency Domains Based On Reinforcement Learning

In modern electronic warfare(EW),the research of anti-jamming game between radar and jammer greatly affects the result of war.Radar should effectively use and control electromagnetic(EM)spectrum to transmit and receive signals.While adversarial jammers can adopt electronic countermeasures(ECM)to disrupt effective use of the EM spectrum by the radar.The game between radar and jammer is now developing in the direction of intelligence and integration.The research on radar anti-jamming game that combines intelligent learning methods and multi-domains anti-jamming technology is particularly important.The game between radar and jammer is not limited to a single domain.They can combine a variety of means in multi-domains to combat with each other.At the same time,EW is developing in the direction of intelligence.EW equipment needs to learn game strategies online in the battlefield.The problem of intelligent radar anti-jamming game in multi-domain can be solved by reinforcement learning.In practice,radars and jammers usually do not know each other’s strategy perfectly.With little prior information,reinforcement learning(RL)provides a solution to sequential decision making problems with unknown environmental models.In the thesis,reinforcement learning is applied to radar anti-jamming via the optimization of radar anti-jamming strategy in time-frequency domain.This work is carried out in two stages: strategy optimization for radar only and game strategy optimization for both radar and jammer.The main contributions are as follows:1.To solve the problem of confrontation between intelligent frequency agile radar and non-intelligent swept frequency jammer with interception function,the intelligent radar anti-jamming game strategy in time and frequency domain based on single agent reinforcement learning is proposed.The following researches are developed:(1)To describe the anti-jamming sequential decision making process,the radar anti-jamming Markov decision process(MDP)model is built.The model includes the state,action,reward and state transition model for radar;(2)To balance between integration efficiency and probability of interception,a flexible adjustable tradeoff between them is devised when defining the reward function of the MDP.(3)The properties of the MDP value function are proved,based on which the optimal anti-jamming strategy under the RL framework is derived.The optimal anti-jamming strategy is solve by Q-learning.2.To solve the problem of confrontation between intelligent frequency agile radar and swept frequency jammer with interception function,the intelligent anti-jamming game strategy of radar and jammer in time and frequency domains based on multi-agent reinforcement learning is proposed.The following researches are developed:(1)To describe the incomplete information and multi-turn game between radar and jammer,the incomplete information static stochastic game model is built.The model includes action,state,reward and type of radar and jammer;(2)In order to solve the problem of incomplete information static stochastic game,the Wo LF-PHC(Win or Learn Fast-Policy Hill Climbing)algorithm of multi-agent reinforcement learning is improved by introducing the property of incomplete information game in it.The game strategy of radar and jammer is obtained using the improved Wo LF-PHC algorithm.The above models and algorithms have been verified by simulation experiments.The results show that the above method can effectively adapt the strategy of radar and jammer to the battlefield.