Low Carbon Operation Of Smart Building Multi Energy System Based On Reinforcement Learning

The buildings and construction industries account for about a third of global energy consumption and carbon emissions,and this figure is expected to increase with urbanization.Therefore,in the context of the continuous progress of the digital reform of the energy system and low-carbon transformation,the smart building energy management has attracted more and more attention.It can effectively reduce the operating cost of energy and improve efficiency,so that the building can achieve the optimal balance in the aspects of energy cost,carbon emissions and user comfort.At the same time,multi-energy coupling and information interaction are deepening in the region,which gradually forms a multi-energy system(MES)of different levels.The MES can not only meet users’ demand for diversified energy use of cold,heat,electricity and gas,but also effectively improve energy utilization efficiency,accelerate the consumption of renewable energy,and promote the sustainable development of energy.Although smart building multienergy system can be a key option for the decarbonization of regional energy systems,research and application challenges arise due to the complexity of the methods required for their modeling and the tools associated with their analysis.Reinforcement learning(RL),as a general-purpose AI technique,has successfully demonstrated its ability to solve a variety of real-world applications and is expected to address these challenges.In this context,based on the goal of reducing building operating costs,this paper adopts the method of deep reinforcement learning(DRL)to start from the low-carbon energy management of a single building,and then carries out research in the aspects of multi-building collaborative optimization in the community and multi-building energy trading in the community,so as to provide a reference for the efficient,reliable and low-carbon operation of the smart building MES.The main research contents and contributions of this paper are as follows:1)A low carbon building energy management method based on DRL is proposed.The smart building energy management system is regarded as an agent in DRL,and the interactive building simulation environment is built for it according to the mathematical model of the building MES.The low carbon energy management problem of buildings is modeled as Markov decision process(MDP),and the state space,action space and reward function are designed for it.The proximal policy optimization algorithm is used to solve the problem.The proposed method can effectively deal with the uncertainty of the system without knowing the building thermal dynamics model clearly and realize the real-time optimal energy scheduling for the combination of multiple energy components in the building.2)A multi-building cooperative optimization method based on federated reinforcement learning(FRL)is proposed.The energy management system of a single building is regarded as an agent in DRL,and the MDP of multi-building collaborative optimization in residential community is designed.A FRL framework is designed for the cooperative optimization problem of multiple buildings in the community.Through the interaction of the intermediate parameters of the DRL model of each building,the training speed of the building energy management system model is improved synergistically on the premise of effectively protecting its data security.3)A peer to peer(P2P)energy trading mechanism based on multi-agent RL is proposed.A community with different types of multi-energy buildings is designed to allow the exchange of electricity between buildings in a P2 P trading platform.The middle market interest rate pricing mechanism has been implemented in the community to fully encourage the buildings to cooperate with each other in energy trading.MDP of P2 P trading is designed and multi-agent proximal policy optimization algorithm is used to solve the problem.The proposed P2 P power trading mechanism helps end users to actively participate in regional green power trading on the basis of making full use of the flexibility of building energy management system,which increases the local consumption of renewable energy,reduces the operating pressure of the grid,and makes the local demand and power generation more balanced.At the same time,the introduction of carbon price can affect the scheduling of MES components by building energy management system,so as to reduce carbon emissions in the community and promote the low-carbon transformation of users.