Reinforcement Learning For Distributed Multi-robot Coordination

As an important research branch of Distributed Artificial Intelligent(DAI),distributed coordination of Multi-Agent Systems(MASs)has become a hot research topic.An MAS consists of a group of interacting autonomous entities,that can be widely applied in enterprise process control,multi-robot systems,resource allocation management,and automated trading.In an MAS,pre-designing the behavior of the agents is often a huge challenge due to the complexity,dynamics,and openness of the environment.As an autonomous learning paradigm based on environmental interaction,Reinforcement Learning(RL)is an effective method to achieve distributed coordination of MASs.Through each agent's interactions with the environment,the goal of coordination can be achieved through the feedback value of the interactions.The thesis carries out a deep investigation on the multi-robot distributed learning methods and models under the framework of Coordination Graphs(CGs),driven by the task requirements of collaborative driving for multi-autonomous vehicles,collaborative search for multi-mobile robots and multi-sensing robot collaborative monitoring,and solves the problems such as learning in a dynamic environment and too simple coordination mechanisms.In particular,this thesis:1.studies distributed collaborative driving problem of multi-autonomous vehicles on highways,and applies the RL technology to the intelligent decision-making of coordinated overtaking and coordinated lane change for multi-autonomous vehicles.Aiming at modeling dynamic change of topology caused by vehicle movement,the thesis proposes a Dynamic Coordination Graphs(DCGs)to model the dynamic dependence of vehicles motion and realize the distributed coordinated learning of multi-autonomous vehicles based on DCGs.The experimental results show that the distributed coordinated learning approaches based on DCGs have better performance in driving safety and traffic system efficiency compared with the individual learning approach and the expert rule-based approach.2.studies the distributed collaborative search problem of Mobile Sensing Robot Team(MSRT)with limited communication capability and observation field,and proposes an online transfer RL approach to realize online transfer and distributed learning of individual interaction experience and learning knowledge under dynamic topology.The experimental results demonstrate that the proposed approach can achieve a better team performance by properly balancing the local individual interests and global team benefits of each agent compared with the independent learning approach.3.studies the distributed collaborative monitoring problem of multi-sensing robots.According to different components(time difference error,learning rate and exploration rate)of coordination mechanism,a variety of distributed coordinated learning approaches based on CGs model are proposed,and the effectiveness and reliability of the proposed approaches are verified in a Distributed Sensor Network(DSN)environment.The simulation experiment results demonstrate that the CGs-based coordinated learning approaches achieves different learning performance by coordinating with different learning components in RL,and have better performance compared with the independent learning approach.