Reinforcement-learning Based Energy Management Strategy For Plug-in Hybrid Electric Vehicles

Facing the pressures of the environment deterioration and energy shortage,new energy vehicle becomes the new trend of the automobile technology development.Plug-in Hybrid Electric Vehicle(PHEV)has become the focus of research due to its good fuel economy and long driving range.Energy management strategy is one of the keys to the vehicle control system.In different working conditions,the output torque distribution and working state of each driving component directly affect the energy efficiency of the hybrid power system.With the continuous development of artificial intelligence technology,it is worth studying to apply the advanced algorithm of intelligent technology to the complex energy management of automobile.In this paper,the application of reinforcement learning algorithm,which has received more and more attention in recent years,in PHEV energy management strategy is studied.Taking the power-split PHEV--Chevrolet Volt as the research object,the vehicle simulation model is built using Autonomie software,the configuration of hybrid power system is introduced,the relationship between torque and speed and energy flow of each driving component under different operating modes is analyzed,and the ideal operating state of each mode is summarized.In order to realize real-time optimal control,this paper proposes an energy management strategy based on reinforcement learning.For rule-based energy management strategies that rely on expert experience and lack of good adaptability to operating conditions,optimization based energy management strategies generally need to predict operating condition information in advance and can only work on offline optimization in most cases.In this paper,we use reinforcement learning algorithm to design the energy management strategy and achieve the near global optimal real-time control under the condition of only relying on the current vehicle operation information.Firstly,according to the analysis of the hybrid power system,the number of control variables by the controller is simplified,and the concept of optimal working curve of the engine is introduced to reduce the freedom of the engine control.The key elements of reinforcement learning algorithm are introduced including state space,action space and reward value.Through simulation experiments,the optimization of the designed energy management strategy based on reinforcement learning is verified.At the same time,the trained controller is tested under other standard driving cycles,and the adaptability of the energy management strategy under other driving cycles is verified.The traditional reinforcement learning algorithm needs to discretize the system state and store the state-action value in the look up table.A neural network is introduced to estimate the state-action value,which can input the system state continuously and increase the dimension of the state space to more completely express the characteristics of the system state.This paper introduces the specific using of neural network,including the inputs and outputs of neural network and the pre-training process.Through the application of temporal difference learning(TD-learning)algorithm,the neural network can train and update the neural network after each trip through the recorded state,action data and updated target value,and finally obtain the neural network close to the real state-action value.Through the training of fixed driving cycle,the optimization performance of the reinforcement learning controller combined with neural network technology is verified.At the same time,through the random driving cycle training,it is verified that the well-trained reinforcement learning controller can achieve good optimization effect in various driving cycles.The energy management strategy based on reinforcement learning proposed in this paper can realize real-time optimal control and further improve the energy utilization efficiency of PHEV.At the same time,the controller can realize online learning,and can adjust the control strategy online for different driving styles and varying driving conditions to achieve adaptive control.