Research On Energy Management Of Commercial Buildings Based On Deeep Reinforcement Learning

The rapid increase of economy and population has led to the accelerated growth of global energy consumption,and the energy crisis has become one of the most important problems facing all countries in the world.Energy shortage causes the imbalance of energy supply and demand,which will lead to the instability of the economic market,so the regulation of energy management is urgent.With the rapid development of urbanization,commercial buildings gradually surpass industry and transportation to become the main source of electricity consumption.It has become one of the most popular topics in intelligent building research to reduce the power consumption of commercial buildings by rationally scheduling the transferable load in commercial buildings.In addition,as commercial buildings are service places,problems related to indoor thermal comfort control in commercial buildings have also become factors that must be considered in the research of intelligent buildings.In this thesis,a commercial building scenario with renewable energy generation,battery storage system,and Heating,Ventilating and Air Conditioning(HVAC)system is considered.The commercial building HVAC and battery storage system scheduling problem under timevarying electricity price,time-varying demand,and time-varying environmental conditions is considered to minimize the building operation cost.Based on the multi-objective joint optimization idea of reinforcement learning,a commercial building energy management algorithm based on deep reinforcement learning is proposed.Through the scheduling of HVAC and battery energy storage systems,the commercial building operation cost is minimized under the condition of effective control of the thermal comfort of the crowd,indoor air quality and the hydraulic balance of the heating network.The main research work and contributions of this paper are as follows:(1)Aiming at the energy management problem of commercial buildings,a commercial building energy management model is established which considers indoor thermal comfort and hydraulic imbalance of heating pipe network in building hydraulic system.This thesis analyzes the energy management problems of commercial buildings under the condition of TOU,timevarying demand and time-varying environment.The multi-objective joint optimization problem of controlling indoor thermal comfort,controlling hydraulic balance of heating network and reducing building operating cost is transformed into Markov decision process,and a strategy for energy management of commercial buildings based on strategy gradient algorithm is proposed.Finally,the effectiveness of the proposed algorithm is verified by simulation experiments.The effect of the power of reward function of thermal comfort deviation penalty on the performance of the algorithm is illustrated by comparative experiments.(2)Based on the energy management model of commercial buildings mentioned above,the HVAC power model is further optimized to decouple the application of the proposed algorithm from regional constraints,and a multi-objective joint optimization problem is proposed to minimize the operating cost of commercial buildings under the condition of controlling the thermal comfort,indoor air quality and the hydraulic imbalance of the heating network.By transforming the above problems into Markov decision process,an energy management algorithm for commercial buildings based on asynchronous advantage actor-critic algorithm is proposed.By controlling the HVAC to satisfy the thermal comfort,the constraints of indoor air quality and the hydraulic balance of heating network,the operating cost of the building is minimized by dispatching the battery energy storage system.The convergence of the proposed algorithm is verified by simulation experiments with the data of real world,and the comparison experiment with baseline strategy proves the effectiveness of the proposed algorithm in reducing operating costs under the condition of satisfying constraint conditions.