Research On Lifetime Maximization Strategy Of Wireless Sensor Networks Based On Reinforcement Learning

In these several years,with the booming of Internet of Things(IoT)technology,Wireless Sensor Networks(WSN)as the underlying technology of IoT have also been developed rapidly.Network lifetime is an important indicator in assessing the performance of wireless sensor networks.Due to the small sensor nodes carrying limited energy,nodes are generally deployed in hazardous areas.The number of nodes deployed these factors causes nodes not easy to replace the battery or recharge,resulting in a limited network lifetime of WSN.The energy consumption of nodes determines the network lifetime,so the technology to reduce the energy consumption of nodes has become a key technology to focus on research in the field of WSN.The current techniques for reducing network energy consumption include hardware optimization,network topology control,node dormancy scheduling,routing protocols,data processing,and node coverage models.In this paper,we address the problem of maximizing the lifetime of wireless sensing networks by studying three aspects of the node coverage model,node sleep scheduling and network topology control,and design a Confidence Information Coverage node sleep scheduling algorithm based on Reinforcement Learning(CICRL).The CICRL algorithm is divided into three phases: the initialization phase,the learning phase and the monitoring phase.The work in this paper focuses on the learning phase and monitoring phase of the CICRL algorithm,which is summarized as follows.(1)In the learning phase of CICRL,this paper designs a node dormancy scheduling strategy based on reinforcement learning.This node scheduling strategy makes three improvements compared to the existing strategy.First,the sensor nodes of the scheduling strategy adopt the excellent Confidence Information Coverage(CIC)model,which has the advantage over the traditional disc model in that it makes full use of the cooperative sensing ability to neighbor nodes and the spatial correlation of environmental variables,and reduces the number of working nodes in each scheduling round without affecting the quality of network service.The number of working nodes in each round of network scheduling is reduced without affecting the quality of service.The second improvement is the action selection strategy in Q-learning,which uses a modified dynamic greedy algorithm to allow nodes to fully explore the action space and not fall into local optimum in the learning process.The third improvement is the introduction of two critical parameters for the payoff formula of Q-learning,namely,the residual energy of nodes and the number of nodes working in the reconstruction region.The improved payoff formula accelerates the Q-learning convergence speed.(2)In the network monitoring phase,for the network topology formed by the scheduling policy in the learning phase,the existing cluster topology technique is improved to enable the efficient transmission of node-aware data.The improvement for the split-cluster topology technique is mainly reflected in the threshold formula for cluster head selection,which constrains the cluster heads with high probability to the nodes with low coverage contribution,balancing and saving the communication energy consumption of the nodes.(3)The superiority and reasonableness of the CICRL algorithm are verified by comparing it with its counterparts in several performance metrics through simulation experiments.The effects of two critical parameters,root-mean-square error and the variance of environmental variables,on the network lifetime of the CICRL algorithm are also tested to guide the selection of the two parameters in practical engineering applications.