Research On Reinforcement Learning Attack And Defense Based On Adversarial Perturbations

Deep reinforcement learning is one of the most attractive research topics in the field of artificial intelligence in recent years.It combines the decision-making ability of reinforcement learning with the perceptual ability of deep learning,achieving an end-to-end learning approach from state input to decision output.Deep reinforcement learning is an iterative method that seeks the optimal policy to maximize the expected reward of the agent.Compared to traditional reinforcement learning algorithms,deep reinforcement learning can handle high-dimensional state inputs and achieve powerful decision control,making it important in solving sequential decision-making problems in the real world.However,existing research has shown that deep reinforcement learning systems are vulnerable to adversarial attacks,which may severely impact their security and even lead to catastrophic consequences.Therefore,this paper studies adversarial perturbation attacks from the perspective of deep reinforcement learning security and proposes feasible and effective defense methods based on this.(1)Regarding attacks,the deep neural network is used to generate adversarial perturbations and added to the original input state of the deep reinforcement learning target model,thereby generating an adversarial state and affecting the decision-making behavior of the target model.Based on this idea,two adversarial attack schemes are proposed:the attack schemes based on generative adversarial networks and the attack schemes based on feedforward generative networks.These two attack schemes perform gray-box adversarial attacks on the target model with limited information,and can quickly generate adversarial perturbations with high attack efficiency.(2)Regarding defense,based on the two types of perturbation attacks proposed in this paper,from the perspective of improving the robustness of the target model against adversarial perturbations,two state pre-processing defense schemes were studied and proposed:the defense scheme based on state stacking noise and the defense scheme based on state reconstruction.These two defense schemes aim to process the adversarial states before they are inputted into the target model,in order to either disrupt the attack performance of the previous perturbation,or to reconstruct the adversarial states with high quality,thereby achieving the defense purpose.The attack and defense schemes proposed in this paper are closely related.The attack scheme uses adversarial perturbation to generate adversarial state to influence the decision behavior of the target model,while the defense scheme enhances the robustness of the target model by preprocessing the adversarial state,thus resisting adversarial attacks.