Energy Management Strategy For Plug-in Hybrid Electric Vehicles Based On SAC Deep Reinforcement Learning Algorithm

As the global environmental pollution problem becomes increasingly serious,the development of energy-saving and environment-friendly new energy vehicles has become the main direction for the development of the automotive industry.Among them,Plug-in Hybrid Electric Vehicle(PHEV)combines the advantages of low emission of pure electric vehicles and long driving range of traditional internal combustion engine vehicles,and has become one of the important development directions in the "three vertical and three horizontal" R&D layout of new energy vehicles in China.As one of the key technologies of PHEV,energy management strategy optimizes the power flow between engine and electric motor to achieve efficient operation of the whole vehicle.However,due to the complex and changing working conditions of the vehicle,it is still a challenging problem to solve the contradiction between the optimal energy consumption and the adaptability of the working conditions for the hybrid vehicle energy management strategy.To this end,this paper designs a vehicle demand torque prediction model based on Long Short-Term Memory(LSTM),proposes an energy management strategy based on Soft Actor-Critic(SAC)deep reinforcement learning with automatic entropy adjustment,and further propose a cloud-based deep reinforcement learning energy management strategy with the help of Cloud Vehicle Collaboration framework.The following research work is carried out in this paper:1.In this paper,we firstly build a vehicle power system model including engine,power battery,electric motor and vehicle longitudinal dynamics equations for a single-axle parallel plug-in hybrid vehicle,and carry out vehicle power performance verification based on AVL CRUISE simulation platform.2.A vehicle demand torque prediction model based on LSTM is designed for the vehicle demand torque prediction problem,and the prediction performance is compared with the Back Propagation(BP)neural network based traditional model.The results show that the prediction accuracy of the designed LSTM-based demand torque prediction model is improved by 86.63%.Based on this,the demand torque dual prediction model based on LSTM neural network is designed to ensure the safety of the vehicle,and the demand torque prediction accuracy is further improved by 44.04%.3.A PHEV energy management strategy based on Soft actor-critic(SAC)deep reinforcement learning algorithm with automatic entropy adjustment is proposed.Specifically,battery state of charge(SOC),the demand torque,acceleration,vehicle speed,and gear information are selected as the states of the strategy to improve fuel economy and avoid overdischarging the battery pack,and the engine torque is used as the output,which effectively improves the adaptability of the strategy to different operating conditions by introducing the automatic entropy adjustment mechanism.To further verify the superiority of the proposed method,it is compared with Deep Deterministic Strategy Gradient(DDPG),fixed entropybased SAC algorithm and Equivalent Consumption Minimum Strategy(ECMS),respectively.The results show that the SAC energy management strategy based on automatic entropy adjustment has the performance of low hyperparameter sensitivity and high self-adaptability,and the fuel economy is improved by 4.37%compared with equivalent consumption minimum strategy,and it has good control effect on battery SOC.4.We further propose a deep reinforcement learning energy management strategy based on the cloud and vehicle-side fusion architecture with the cloud computing performance and engine characteristics in mind.Based on the sliding window principle to reduce the amount of cloud data upload,the engine start-stop frequency and the amount of torque change in adjacent moments,the reasonable distribution of engine operating points in the efficient region is achieved based on the expert’s empirical knowledge,and the simulation is verified under real working conditions.The results show that the proposed method can effectively realize the cloud-vehicle interaction and achieve the optimization goal of reducing the startstop frequency of the engine.