Research On Indoor Environment Control Technology Based On Deep Reinforcement Learning

The indoor environment of buildings is complex and changing,and it is important to maintain the stability of indoor environmental parameters,which not only improves the quality of people’s life and work,but also plays an important role in protecting indoor objects and reducing energy consumption.Currently most indoor environment management systems have only monitored without control,and most of the control aspects are based on single-point monitoring of the indoor environment with a centralized control method,which is difficult to ensure the uniformity and stability of indoor environmental parameters.To address these challenges,this paper investigates two aspects of indoor environment intelligent control and prediction,and derives a digital twin control model,the main contents of the paper are as follows:Firstly,an intelligent distributed environmental control system based on model-free deep reinforcement learning is proposed.The system consists of three components:an intelligent controller,distributed facilities,and distributed sensors.The distributed sensors are used to monitor the environmental parameters in real time.And the RH-rainbow algorithm is deployed in the distributed device with humidity environment control as an example.In RH-rainbow,the reward consists of the average absolute difference of humidity and the energy consumption of the distributed facilities.The action of the agent is the humidity and fan settings of the constant humidity machine.The algorithm is evaluated and compared in two scenarios with different air outlets,different number of sensors,different reporting intervals of sensors and interference patterns with external wind masses.The results show that RH-rainbow outperforms the fixed and conditional control strategies,Deep Q-Network and ProportionIntegration-Differentiation(PID)at the combined level of anti-interference ability and energy consumption.Secondly,a prediction model is proposed for environmental parameters of temperature and humidity by indoor distributed sensors based on spatiotemporal adaptive graph convolution.The model windows the original data to generate a bunch of sub sequences which are put into the model for training.The normalized adaptive adjacency matrix is used to automatically learn the position correlation between each node in the space,the gate recurrent unit is used to learn the time correlation in the subsequence,and finally the node embedding matrix is used to learn the relationship between multiple features,including the coupling of temperature and humidity,and the impact of external weather on indoor environmental parameters.The model has been verified in multiple data sets,and the prediction effect of MAGCRN is the best.Then the predictive model is fused with the conditional control strategy and evaluated in two scenarios.The control effect is better than the traditional PID.Finally,a digital twin control model is proposed to reduce the time of model training and the energy consumption of servers and facilities.The spatio-temporal adaptive graph convolution prediction algorithm is used to learn the historical temperature and humidity data and generate a digital temperature and humidity prediction model.Combining predictive models with deep reinforcement learning intelligent distributed environmental control algorithms to generate digital temperature and humidity control models.The control model is validated in a real scenario and the final virtualized data is similar to the real world.