Research On Decision-making Control Of Intelligent Vehicle Based On Deep Reinforcement Learning Of Model Integration

With the continuous improvement of computer computing performance,artificial intelligence has been greatly developed,and it has performed well in many fields.Intelligent vehicle is the direction of vehicle development,and its decision-making control is one of the most important key technologies.To achieve the popularization of intelligent vehicle,we must solve its decision-making control problem.In the field of intelligent vehicle decision-making control,deep reinforcement learning(DRL)has many advantages over the traditional rule-based method.The decision-making control strategy based on rules is quite tedious,and it is difficult to consider all the problems in the complex driving environment,so its adaptability is poor,and DRL algorithm can avoid these problems.In this paper,DRL algorithm is applied to the research of intelligent vehicle technology to solve the decision-making control problem of intelligent vehicle in continuous action space.Firstly,the ECDDPG algorithm which can be applied in the field of intelligent vehicle decision control is designed.This paper analyzes how human beings make decisions on the current driving environment in the driving process,and compares the similarities and differences between DRL algorithm and DRL algorithm in decision-making performance.The design of the algorithm is based on the algorithm framework of DDPG.Aiming at the disadvantages of unstable training process,long training time and slow convergence speed,some improvements are made.The vehicle dynamics model is introduced to judge the rationality of the samples generated in the process of agent interaction with the environment.The experience playback pool is classified to store different kinds of experience samples,and the agent will learn from them.After learning,the strategy will avoid unreasonable and dangerous actions.In order to avoid the local optimization of the strategy,we prioritize the generated experience samples to learn high-quality experience,improve the learning efficiency,and reduce the priority of the repeated learning samples.A hierarchical decision control method in complex environment is proposed.Taking overtaking command as an example,the driving action decision module based on DQN is modeled.Secondly,it compares a variety of driving simulation software platforms which can be used to verify DRL algorithm,analyzes and compares the characteristics of various software,and finally selects TORCS software as the simulation environment of this paper.The use of TORCS software,communication method,interface design,data format collected by built-in sensors,and action instructions of vehicle operation are described in detail.The system architecture of DRL algorithm and software simulation is proposed.In this paper,the construction of simulation platform and the design of algorithm are described in detail,including the hardware and software environment of simulation,the design of neural network,the design of return function.The environment data information of the simulation task is analyzed in detail,and the return function form of multiple index items accumulation is proposed.The intelligent experience takes action by maximizing the return function value.Finally,the correlation analysis of the experimental results is carried out.The round average return of ECDDPG is more stable than that of DDPG in the training process,and the convergence speed is faster.Compared with the original DDPG,the efficiency of ECDDPG is improved by 27%.The generalization performance of the proposed algorithm is analyzed.By replacing the runway environment with untrained environment,the algorithm controls the vehicle to run the whole course,which shows that the algorithm has good generalization.Under the control of the strategy,the overtaking is completed safely.The results show that the algorithm can be applied to the decision-making control of intelligent vehicle.