Deep Reinforcement Learning Approaches For Content-Cached Placement And Delivery Policy In Wireless Networks

With the increasing functionality of mobile devices and the rapid development of social networks,mobile data traffic is expected to grow exponentially in the coming years.Some popular content files will be forwarded and watched in large quantities in a short period of time.During the peak period of communication,it will easily cause communication delays and interruptions,which will greatly reduce the quality of user experience.The Combination of D2 D communication technology and wireless caching technology is applied in this paper,which makes the content closer to the user and provides a new direction for realizing the reliable link transmission and reducing the cost of backhaul links.In a cache-enabled D2 D communication network,which content to cache is critical for improving the cache hit rate of the network.If cached content is placed properly,each user can easily find their requested content locally or by establishing D2 D communication with nearby users to achieve high throughput.However,the content storage capacity of mobile devices is limited,the mobility mode is uncertain,and the popularity distribution of the requested content is unknown a priori.In addition,multiple users may request the same content simultaneously,or that one content may satisfy multiple request users simultaneously in cache-enabled D2 D networks,how to achieve efficient content-cached placement and content-cached delivery is the main technical problem for cache-enabled D2 D network resource allocation.For the problem of cache resource allocation of the cache-enabled D2 D communication networks in next-generation wireless communication networks,we study a joint content-cached placement and content-cached delivery policy.Recurrent neural network and deep reinforcement learning algorithm are respectively used to solve the problem of which content the terminal device caches and the dynamic decision problem of selecting the sender and receiver during the content delivery phase,which significantly improves the cache hit ratio of the whole network system,and reduces the energy consumption and the cache content transmission delay.The main research contents are as follows:(1)In this paper,we study the wireless caching technology and D2 D communication technology.We analyze the application of recurrent neural network algorithms and deep reinforcement learning algorithms in the communication field,and propose a novel joint content-cached placement and content-cached delivery strategy to solve the device access competition in the network.(2)For the questions of which content the terminal device caches and where it is cached,an algorithm based on recurrent neural network is proposed to implement the placement of the cached content.The long-short term memory network(LSTM)algorithm and echo state network(ESNs)algorithm are respectively used to predict user mobility and content popularity.The prediction accuracy of these two algorithms is compared by simulation,and it is verified that the content-cached placement strategy can effectively improve the cache hit ratio.(3)When the local cache of the user cannot satisfy its own request,the user may consider establishing a D2 D link with the neighboring user to implement the content delivery.In order to decide which user will be selected to establish the D2 D link,we propose the novel schemes based on deep reinforcement learning to implement the dynamic decision-making and optimization of the content delivery problems,aiming at minimizing long-term delay and consumption,thereby relieving the traffic pressure of the core network.In addition,a performance compromise is achieved by adjusting the weight coefficient of the reward function.The simulation results show that the perceptual decision-making ability of the deep reinforcement learning algorithm can effectively solve the dynamic decision problem of selecting the sender and receiver in the content delivery stage of the cache-enabled D2 D communication network.Through comparative analysis,the value-based deep Q-Learning Network(DQN)algorithm has lower transmission delay and energy consumption.