Research On Reinforcement Learning Exploration Strategies Based On Feature Embedding

In recent years,deep reinforcement learning has achieved excellent results in a wide range of fields,but the low sample efficiency of deep reinforcement learning is still a problem to be solved urgently.In traditional experimental environments,agents usually use fixed exploration mechanisms to improve sample efficiency.However,when the state space is too large or the external rewards are sparse,this method of exploration cannot achieve the ideal effect.In order to solve the above problems,this paper combined the existing deep reinforcement learning exploration methods to analyze the difficulties of high-dimensional exploration problems in detail,and proposed two different high-dimensional reinforcement learning exploration algorithms: Count-based Exploration via Embedded State Space for Deep Reinforcement Learning and Explore in Embedded State Space with Random Network Distillation.Based on the fact that visual observations may contain irrelevant information,some features in the state space are not valid for an agent.From this point of view,we propose Count-based Exploration via Embedded State Space for Deep Reinforcement Learning(CEESS)which uses a simple and efficient way to extend counting exploration method to high-dimensional reinforcement learning environment.By training a deep neural network consisting of two sub-modules,a feature extractor is obtained which can better extract state features and can be applied to different environments.In addition,this paper also proposes Explore in Embedded State Space with Random Network Distillation(E2S2RND),which improves the Random Network Distillation(RND)and adds an optimized feature extraction module to improve the robustness of the RND to white noise problems.In future work,we will continue our research on reinforcement learning exploration algorithms.Extend the reinforcement learning framework of auxiliary agents to more stochastic reinforcement learning environments.At the same time,combined with other related research to find a better state feature representation to improve the exploration ability of the agent.