Learning In Cooperative Multi-agent Team

As one of the new technologies in the Distributed Artificial Intelligence area, the multi-agent technology has already filtered into many taches of the information society. The multi-agent learning technology has received increasing attention in the Distributed Artificial Intelligence community, which is a young and important intersection area constituted by Distributed Artificial Intelligence and Machine Learning.The research topic of this dissertation has been focused on cooperative multi-agent teams. The research content is composed of three parts as the following: 1) the cooperative problem solving process involved multiple agents; 2) the multi-agent cooperative reinforcement learning methods by using reinforcement learning theory, shared experiences and Markov Game theory; 3).the other learning methods which can improve cooperative capability of multi-agent teams.The primary work and main contributions of this dissertation include:1. A new cooperation framework for multi-agent team is proposed, and a prototype system named MBOS is presented. The cooperation framework is based on Teamwork theory and the rational BDI model, which presents the whole cooperative problem solving process from the team formation to the result evaluation. In the implementation of MBOS, an agent architecture named AGENTFRAME and its corresponding mechanisms including a multiple threads control mechanism and a conversation management method are proposed. The multiple threads control mechanism can make agents to fulfill decision and cooperation tasks effectively; the conversation management method consists of establishing three interaction protocols (Cooperate-Protocol, Negotiate-Protocol and Employ-Protocol), representing state transition process by Finite State Machine and constructing relevant message process flows, so as to realize an efficient conversation. MBOS has been tested and evaluated by several application instances. The result shows that AGENTFRAME is a feasible agent architecture, and the agent based on this kind of architecture is able to run persistently and accomplish many intelligent behaviors such as reasoning and cooperation.2. By importing reinforcement learning method into learning process of multi-agent team, a shared experience tuples multi-agent cooperative reinforcement learning method (SE-MACOL) is proposed. A role assignment algorithm based on bipartite graph is proposed, which is obviously efficient proved by applications. A knowledge representation form composed of sequential pair as is proposed, in which state-value and action-value are represented as numerical values like Euclid Distance, and the space of state-action is reduced by incorporating many similar states and actions. The algorithm of SE-MACOL makes experience tuples be shared with other agents through similarity transformation according to homogeneous subtasks. In order to apply and verify the learning method, a series of experiments are done, whose results show that cooperation efficiency of the team is improved obviously.3. A Team Markov Game based multi-agent cooperative reinforcement learning method (TMG-MACOL) is proposed. An evaluating method of phase game based on long-time payoff matrix is proposed, in which the matrix will gradually converge at a stable value by constant interaction with environment and payments from it. An action select strategy based on the process of fictitious play is proposed, which can drive the agent to take the optimal action. A series of experiments are done to apply the learning method, which has educed convergence and showed the effectivity of the learning method. Also, a useful conclusion about influence factorsof stable value and speed of convergence is drawn by experiments.4. Other new learning methods in validation process of team structure and teamwork process in the cooperative multi-agent team are explored. Aimed at the validation process of team structure, a case based learning method fitted for multi-agent cooperation is proposed, in which a case structure composed of three...