| In recent years,deep reinforcement learning has experienced rapid development and has been widely applied in various fields such as autonomous driving,robotics,industrial control,etc.However,research has shown that current deep reinforcement learning methods have privacy and security vulnerabilities.Therefore,this study conducts in-depth research on the privacy and security issues of deep reinforcement learning and analyzes them from an attack perspective.The study reveals that current privacy attacks on reinforcement learning models are difficult to extend to real-world scenarios and there is a lack of exploration on privacy threats to reinforcement learning models,leaving undisclosed privacy issues.This study focuses on deep reinforcement learning policy models and proposes a privacy attack framework based on the decision sequences of deep reinforcement learning policy models.The framework utilizes high-dimensional policy features hidden in decision sequences to identify the observation data features of the model,thereby inferring privacy information.Considering the observation data distribution and observation variables,two specific privacy attacks are derived:observation data distribution inference attack and observation variable inference attack,achieving high inference accuracy.(1)To address the privacy leakage issue of the observation data distribution in deep reinforcement learning models,a novel privacy attack method is proposed based on the decision sequences in the form of state-action pairs.Through the research on the privacy leakage issue of the observation data distribution in deep reinforcement learning models,it is found that the current attack schemes based on the model’s reward mean and variance as features lack scalability and feasibility.Therefore,a method is proposed that infers the observation data distribution by utilizing the high-dimensional policy features hidden in the decision sequences,enabling privacy attacks.The feasibility of the proposed method is validated through experiments in benchmark reinforcement learning environments,demonstrating its superior attack accuracy.(2)To address the privacy leakage issue of the observation variable set in deep reinforcement learning models,a privacy attack method based on the model’s action sequences is proposed.Through the analysis of the practical application scenarios of deep reinforcement learning models,the overlooked inherent risks are identified,and the information of the model’s observation variable set is considered as privacy information of value.Under stronger constraints than observation data distribution inference,this study proposes an observation variable set inference method based on the model’s action sequences,utilizing the action sequence features brought by the observation variable set to infer the privacy information of the observation variable set.The feasibility of the framework is validated through experiments,achieving good inference performance. |
