Multi-agent Reinforcement Learning Based Multi-path Routing Protocol For Wireless Sensor Networks

Wireless sensor networks are application-specific networks, in which data gathered by sensor nodes in different cases have different quality of service requirements, which increases the complexity of routing protocol designing. In some existing multipath routing protocols, data that have high priority are transmitted through shortest paths to reduce transmission delays. However, due to the highly dynamic network topology, several shortest paths that have the same hops may exist in the network. Thus, it is an interesting topic to find an optimal route from these shortest paths to support QoS demands for the transmission of high priority data.In this thesis, we propose multi-agent reinforcement learning based multipath routing protocol by viewing the wireless sensor network as a multi-agent system and each sensor node in the network as an independently learning agent. In the protocol, data priority is firstly considered before data transmission. If the data to be transmitted has low priority, then it is sent to a neighbor node selected randomly from the route table. Nevertheless, if the data has high priority, then queue length of the neighboring nodes and link quality between sending node and neighboring nodes are taken into consideration during route selection. Routing of the high-priority data is modeled as a Markov Decision Process (MDP) by employing multi-agent reinforcement learning theory. Meanwhile, by exchanging interaction information with direct neighbors, a distributed value function based Q-learning algorithm is presented. Then, we conduct several simulation experiments with OMNeT, and results show that our proposed algorithm has a lower transmission delay and a higher packet delivery ratio during the transmission of high priority data, and the network traffic is also balanced.We further consider the "curse of dimensionality" problem in the learning process, and propose a state abstraction based multipath routing protocol for wireless sensor networks. In the protocol, queue lengths of the neighbor nodes are sorted and the ranks are utilized as part of the state vector to decrease the size of state space. Simulation experiments demonstrate that the algorithm after state abstraction learns faster and has better network performances.