Research On Electric Vehicle Charging And Discharging Control Strategies Based On Deep Transfer Reinforcement Learning

In recent years,the population density in Chinese cities is set to increase further.In the context of sustainable energy development and significant emission reductions,the infrastructure for charging poles has been listed as one of the "seven new infrastructures".Developing electric vehicles and new energy is the trend for Chinese energy strategy and green transportation,which poses new challenges and higher demands for large-scale and flexible access of electric vehicles to the power grid.Electric vehicles have the characteristic of mobile energy storage,and the behavior of users,the charging status of electric vehicles,and charging prices all have strong uncertainty,greatly increasing the complexity of electric vehicle charging energy scheduling.This poses a challenge for the rapid development of effective charging control strategies in different scenarios.Therefore,this study proposes an electric vehicle charging/discharging control strategy based on deep transfer reinforcement learning(DTRL).The specific work of this study is as follows:(1)Firstly,the charging loads characteristics of electric vehicles are analyzed from multiple perspectives such as the type of electric vehicle,battery characteristics,charging/discharging prices,and weather conditions.The travel behavior of electric vehicle users in residential areas,work areas,and other areas are described,analyzing the charging loads characteristics of electric vehicles in different regions.Based on analysis of real-world electric vehicle driving data,the general rules of private car travel behavior in different areas are summarized,and a mathematical model of charging/discharging that is consistent with electric vehicle load characteristics is established.(2)The charging/discharging control problem of electric vehicles is described as a dynamic decision problem,and a Markov decision process(MDP)for charging/discharging electric vehicles is established with the objective of minimizing charging costs and meeting user charging demands.Considering the uncertainty of electric vehicle travel behavior and charging prices,an electric vehicle charging/discharging control strategy based on deep deterministic policy gradients(DDPG)is proposed.Simulation results show that the charging/discharging control strategy based on DDPG can select continuous charging/discharging power based on the system environment status to achieve real-time response to dynamic charging prices and reduce charging costs while meeting user charging demands.(3)Under new charging scenarios(such as changes in charging areas and time),changes in electric vehicle user travel behavior and charging prices will cause the training-based charging/discharging control strategy based on deep reinforcement learning to no longer be applicable.In addition,developing a new charging/discharging control strategy based on deep reinforcement learning for new scenarios requires a lot of time and data samples.To quickly develop a deep reinforcement learning-based charging/discharging control strategy for new scenarios,this study proposes a DTRL method: transfer in deep deterministic policy gradients(TDDPG).Firstly,the electric vehicle charging/discharging control strategy based on DDPG is fully trained in scenarios where data is sufficient.Then,the TDDPG algorithm is used to transfer the fully trained charging/discharging control strategy to the new scenario.This method differs from traditional Actor-Critic(AC)methods by adding a value network to accurately evaluate the electric vehicle charging/discharging control strategy in the target task.Simulation results show that this method has good performance in electric vehicle charging/discharging control,significantly shortening the development cycle of electric vehicle charging/discharging strategies in different scenarios and reducing outliers that meet user charging demands.