Information Fusion Methods For Heterogeneous Agents In Multi-Agent Reinforcement Learning

The rapid development of artificial intelligence technology has led to numerous autonomous intelligent systems that are widely applied in various fields of human society.Individual intelligent agents often struggle to cope with complex real-world tasks.Therefore,multi-agent systems have become one of the research hotspots in the field of artificial intelligence.Multi-agent reinforcement learning aims to enable multiple agents to achieve common goals in complex environments through interaction and collaboration.Centralized training with decentralized execution is an efficient mode widely applied in multi-agent reinforcement learning.However,this mode leads to suboptimal decisionmaking performance since agents do not fully utilize team information during execution.Moreover,existing multi-agent reinforcement learning methods often adopt parameter sharing without fully considering the heterogeneity among different types of agents,resulting in negative transfer.To address these issues,this thesis conducts the following research from the perspective of information fusion:(1)To address the issue of suboptimal decision-making caused by the failure of agents to fully utilize team information during distributed execution,this thesis proposes the QMIX-GNN model.This model can effectively leverage the information fusion ability of graph neural networks to enhance agents’ perception and adaptation to team states during execution.Moreover,it uses a transformation matrix and a tower structure to handle heterogeneous agents with inconsistent observation and action spaces.Comparative experiments on a public platform demonstrate the superior performance of QMIX-GNN in multi-agent scenarios.(2)To address the negative transfer caused by parameter sharing among heterogeneous multi-agent systems,this thesis proposes the QMIX-HMM model.This model further employs a three-layer feature fusion mechanism that uses graph neural networks,multi-gate mixture experts,and bilinear pooling to address the negative transfer problem caused by parameter sharing among heterogeneous multi-agent systems and more efficiently utilize team information.Comparative experiments on a public platform demonstrate the superior performance of QMIX-HMM in multi-agent scenarios with high heterogeneity.(3)Based on the QMIX-HMM model,this thesis designs and implements a business process recommendation system that can mine and utilize business process information to assist users in quickly building business processes.This thesis elaborates on the system’s design details and verifies the system’s meeting of design goals through testing and demonstration.