Research On Energy Management Strategy Of Hybrid Electric Vehicle Based On Deep Q Learning

With energy shortage,environmental degradation,and global warming,the emergence of new energy vehicles has effectively alleviated global environmental and energy problems.Hybrid vehicles can solve the problems of traditional fuel vehicle emissions and pure electric vehicle mileage anxiety,and are an important transition product from internal combustion engine models to pure electric vehicle models.However,as car ownership increases,leading to increased complexity of road traffic,so the impact of traffic factors on vehicle fuel economy will also increase.In recent years,artificial intelligence technology and information and communication technology have been empowering the automotive industry to design energy management strategies(EMS)for hybrid vehicles through intelligent technology to coordinate the power distribution problem between two different power sources.In this paper,two types of deep reinforcement learning are used to solve the energy management problem of hybrid vehicles with series hybrid vehicles as the research object.The specific analysis of the convergence,working condition adaptability and equivalent fuel consumption of the proposed strategy is carried out and compared with the energy management strategy based on dynamic programming:Firstly,three vehicle configurations of hybrid vehicles are elaborated and several operating modes of tandem hybrid vehicles are analyzed,and the parameters of the main components of the power system are determined by the formula calculation.Based on the engine and generator static experimental data,the optimal operating line of the engine-generator unit is fitted by interpolation operation and least squares method;the equivalent circuit model of the battery is established,and the operating range of the battery is determined based on the curves of the open circuit voltage and charge/discharge internal resistance with SOC;the mathematical model of the powertrain and energy management problems of the whole vehicle is established.In order to collect working conditions that are more in line with drivers’ actual off-duty commute,the layout of the road network and the control information of intersection signals are obtained through Baidu maps and manual fieldwork,as well as vehicle identification and statistics of actual traffic flow through YOLOv5-deepsort-based algorithms.The traffic flow model of the experimental road section is constructed by VISSIM simulation software,and the vehicle speed is collected to form a data set of actual vehicle conditions,which is used to prepare the data for the energy management strategy in the later paper.In order to compare and analyze with the proposed strategy,the EMS based on dynamic programming is proposed,and the effects of dispersion accuracy of state variables and control variables on the control effect are analyzed,and the appropriate dispersion accuracy is selected and simulated under NEDC conditions.Then,the Q-Learning algorithm(Q-Learning)is only suitable for low-dimensional state problems because of the explosion of Q-value table dimensions due to the need for discretization of both states-actions,which leads to inefficient computation and difficult convergence with a small number of samples.We propose to combine Q-learning with Deep Learning(DL)to form Deep Q Network(DQN)and apply it to energy management problems.By using deep neural network as Q-approximation function instead of Q-value table to obtain Q-values in Q-learning,we can improve the capability of continuous or high-dimensional state processing.The empirical replay mechanism and the inclusion of a target network are introduced to reduce the temporal correlation between sample data and improve the stability of the learning process.Simulation results show that the proposed strategy differs from the optimal fuel economy by 7.36% and 7.30% under NEDC and UDDS offline training conditions,respectively,and DQN-EMS is able to maintain SOC and reduce equivalent fuel consumption better.The offline training of the intelligent body under NEDC condition is applied online to WLTC as an unknown condition,and the difference with the optimal fuel economy is 9.38%,which shows that the proposed strategy has some adaptability to the working condition.Finally,to address the problem of overestimation caused by directly selecting the action that maximizes the Q value after getting all the action Q values corresponding to the state in the DQN algorithm,and after analyzing the reasons for generating overestimation,we combine the idea of Double Q-Learning to form the DDQN algorithm(Double Deep Q Network,DDQN)by decoupling the selection action from the evaluation action,using Different network parameters are used for the selection action and evaluation action to solve the overestimation problem in DQN.Simulation analysis is also performed and the results show that the difference with the optimal fuel economy is 6.09%,6.42% and 7.47%,which is 1.18% and 0.813% and1.74% better than DQN-EMS,respectively,with better economy and improved convergence speed.For the commuting conditions,the collected actual commuting conditions are used as the training data set of DDQN-EMS to train the intelligent body offline,and the network parameters and weight factors are adjusted for multiple training to make it converge.The debugged parameters are used for simulation analysis,and the results show that the vehicle can directly adjust the energy distribution according to the vehicle driving conditions and the vehicle’s own state and other information to meet the design requirements.