Research On Cluster Obstacle Avoidance Algorithm Based On Q-learning

In recent years,unmanned aerial vehicles(UAVs),unmanned cars and other intelligent agents have developed rapidly due to their high stability,strong adaptability and low risk.Clustering of agent solves the problem of limited function of individual agent,and gives full play to the advantages of clustering while effectively integrating intelligent individuals.Cluster obstacle avoidance has always been an important module of swarm control for agents.Most obstacle avoidance algorithms tend to fall into local optimal values when enacting complex obstacles,which makes it difficult for agents to avoid obstacles quickly.According to the needs of the agent cluster in the obstacle avoidance process of task execution in the obstacle environment,this thesis combined the Flocking cooperative control algorithm and Q-Learning algorithm and proposed two obstacle avoidance algorithm models,which solved the obstacle avoidance problem of the agent in task execution and improved the agent cluster training algorithm.The main research work of this thesis is as follows:(1)Aiming at maintaining a safe distance between cluster individuals while avoiding obstacles,this thesis adopts Flocking algorithm as the control algorithm of the cluster and introduces ?-agent and ?-agent modules in the algorithm to construct potential energy field function and virtual structure points to guide cluster individuals to maintain a stable moving distance.(2)The problem of structural differences of formation types in the process of obstacle avoidance is analyzed,the formation type database is established according to the requirements of cluster formation,and the evaluation criteria for obstacle avoidance are put forward to quantify the parameters such as convergence time,distance cost,and formation structure difference in the process of obstacle avoidance.(3)The obstacle avoidance model of the agent cluster in complex obstacle environment is designed,and the obstacle avoidance factor is proposed to realize the adaptive selection of obstacle avoidance strategies in different obstacle environment.The corresponding motion model is constructed based on the control requirements of the agent under different obstacle avoidance strategies.(4)The Q-Mutual algorithm,which is suitable for the training of agents in clusters,is proposed in this thesis.According to the threshold interactive Q-learning algorithm,the communication volume between agents is reduced,and the interactive training of clusters and the distributed autonomous decision-making of agents are realized.In this thesis,a team change obstacle avoidance model and an agent autonomous cooperative obstacle avoidance model were proposed.The potential field function between the Flocking control algorithm was constructed and combined with Q-learning algorithm to improve the movement model of the agent.The proposed Q-Mutual algorithm has been studied and trained in specific scenes and random obstacle environments,and different training algorithms have been analyzed by data fitting.It is verified that the improved algorithm improves the efficiency of obstacle avoidance effect,convergence rate and other aspects,and realizes the stability of the agent in the complex obstacle environment.Finally,the feasibility and effectiveness of the obstacle avoidance model in practical application are verified by the flight control simulation experiment of the quadrotor UAV.