Research On Edge Caching Strategy Based On Deep Reinforcement Learning

As the number of network devices continues to grow and data consumption application services flourish,network traffic is growing at an exponential rate,exacerbating source server pressure and link traffic load.Most of these data are duplicate content requested by users.Edge caching technology can cache this content on edge servers closer to users,reducing redundant data transmission in the core network to ease traffic congestion and reduce latency,greatly improving the quality of service for users.Given the limited caching resources of edge servers,how to effectively utilize the cache space to reduce the transmission delay of content and improve the hit rate becomes a key issue for edge caching.Therefore,this paper studies edge caching policies based on deep reinforcement learning algorithms,and the main work is as follows:(1)A deep reinforcement learning-based edge caching policy is proposed to address the problems of traditional caching policies based on a single factor and lack of dynamism.Firstly,the transmission delay is represented in the cloud edge-end architecture,and the difference between its before and after moments is defined as the positive and negative rewards of the system,and the caching problem is modeled as a Markov decision process,then the edge caching policy based on deep deterministic policy gradient is proposed and its parameter update is improved with the aim of minimizing the transmission delay.The final simulation experimental results show that the proposed caching strategy outperforms other conventional algorithms in terms of delay and hit rate.(2)An edge caching strategy based on multi-intelligent deep reinforcement learning is proposed to address the problem of single-intelligent body not being able to cooperate in caching in large-scale edge networks.The global transmission delay is first represented in the software-defined network architecture,and the cooperative caching problem is modeled as a partially observable Markov decision process,and then the edge caching strategy based on multi-intelligent deep reinforcement learning is proposed with the aim of minimizing the global transmission delay.Information encoding and feature extraction modules are introduced in the algorithm framework to reduce the multi-intelligence observation dimension and extract the temporal features so that the intelligences can obtain more effective observations.To address the problem of unreasonable ratio of observation information among multiple intelligences,the attention mechanism is introduced into the multi-intelligence deep reinforcement learning method.Through the similarity relationship among the intelligences,the attention network assigns different weights to the weighted sum as observations to avoid the undifferentiated acquisition of other intelligences’observations,thus obtaining a more comprehensive input,emphasizing the key information and filtering the redundant information to improve the decision-making efficiency of the intelligences.Finally simulation experiments verify that the proposed algorithm in this paper can make better decisions in terms of delay and hit rate.