Building energy system with energy data simulation for pre-training predictive building models

1. A system for controlling heating, ventilation, or air conditioning (HVAC) equipment of a building, the system comprising one or more processing circuits configured to:

generate simulated building data using a simulation model of the building, the simulation model of the building is configured to simulate building conditions and energy consumption of the building based on simulated weather data or historical weather data for a first time period prior to deploying a reinforcement learning (RL) model for the building;

pre-train the RL model using the simulated building data;

operate the HVAC equipment of the building by using the RL model to select control actions for the HVAC equipment based on actual weather data for a second time period after deploying the RL model for the building; and

retrain the RL model using actual building data generated responsive to operating the HVAC equipment using the RL model.

2. The system of claim 1 , wherein the simulation model of the building is configured to simulate building conditions and energy consumption of the building based on simulated control actions for the HVAC equipment.

3. The system of claim 1 , wherein:

the simulated building data comprise a set of states and corresponding simulated control actions for the HVAC equipment; and

pre-training the RL model using the simulated building data comprises training the RL model to select the control actions based on a current state and a reward policy.

4. The system of claim 1 , wherein pre-training the RL model comprises training the RL model to maximize a reward based on at least one of energy consumption of the building or comfort of building occupants.

5. The system of claim 1 , wherein:

pre-training the RL model using the simulated building data is performed prior to deploying the RL model in a building; and

operating the HVAC equipment of the building using the RL model and retraining the RL model using the actual building data are performed by a building controller of the building which receives the RL model after the pre-training.

6. The system of claim 1 , wherein:

the HVAC equipment of the building comprise at least one of an air handling unit (AHU) or a rooftop unit (RTU); and

the control actions are for the AHU or the RTU.

7. The system of claim 1 , wherein the RL model comprises at least one of a Q-learning model, a support vector machine, an artificial neural network (ANN), a convolutional neural network (CNN), a recurrent neural networks (RNN), a deep learning model, a supervised machine learning model, or an unsupervised machine learning model.

8. A method for controlling heating, ventilation, or air conditioning (HVAC) equipment of a building, the method comprising:

generating simulated building data using a simulation model of the building;

pre-training a reinforcement learning (RL) model using the simulated building data prior to deploying the RL model in the building;

operating, by a building controller which receives the RL model after the pre-training, the HVAC equipment of the building using the RL model; and

retraining, by the building controller, the RL model using actual building data generated responsive to operating the HVAC equipment using the RL model.

9. The method of claim 8 , wherein generating the simulated building data comprises using the simulation model of the building to simulate building conditions and energy consumption of the building based on simulated control actions for the HVAC equipment.

10. The method of claim 8 , wherein:

the simulated building data comprise a set of states and corresponding simulated control actions for the HVAC equipment; and

pre-training the RL model using the simulated building data comprises training the RL model to select control actions based on a current state and a reward policy.

11. The method of claim 8 , wherein pre-training the RL model comprises training the RL model to maximize a reward based on at least one of energy consumption of the building or comfort of building occupants.

12. The method of claim 8 , wherein:

generating the simulated building data comprises using the simulation model of the building to simulate building conditions and energy consumption of the building based on simulated weather data or historical weather data for a first time period prior to deploying the RL model in the building; and

operating the HVAC equipment comprises using the RL model to select control actions for the HVAC equipment based on actual weather data for a second time period after deploying the RL model in the building.

13. The method of claim 8 , wherein:

the HVAC equipment of the building comprise at least one of an air handling unit (AHU) or a rooftop unit (RTU); and

operating the HVAC equipment of the building using the RL model comprises using the RL model to select control actions for the AHU or the RTU.

14. The method of claim 8 , wherein the RL model comprises at least one of a Q-learning model, a support vector machine, an artificial neural network (ANN), a convolutional neural network (CNN), a recurrent neural networks (RNN), a deep learning model, a supervised machine learning model, or an unsupervised machine learning model.

15. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

generating simulated building data using a simulation model of the building, the simulation model of the building is configured to simulate building conditions and energy consumption of the building based on simulated weather data or historical weather data for a first time period prior to deploying a reinforcement learning (RL) model for the building;

pre-training the reinforcement learning (RL) model using the simulated building data;

operating the HVAC equipment of the building by using the RL model to select control actions for the HVAC equipment based on actual weather data for a second time period after deploying the RL model for the building; and

retraining the RL model using actual building data generated responsive to operating the HVAC equipment using the RL model.

16. The one or more non-transitory computer-readable storage media of claim 15 , wherein generating the simulated building data comprises using the simulation model of the building to simulate building conditions and energy consumption of the building based on simulated control actions for the HVAC equipment.

17. The one or more non-transitory computer-readable storage media of claim 15 , wherein:

the simulated building data comprise a set of states and corresponding simulated control actions for the HVAC equipment; and

pre-training the RL model using the simulated building data comprises training the RL model to select the control actions based on a current state and a reward policy.

18. The one or more non-transitory computer-readable storage media of claim 15 , wherein:

pre-training the RL model using the simulated building data is performed prior to deploying the RL model in a building; and

operating the HVAC equipment of the building using the RL model and retraining the RL model using the actual building data are performed by a building controller of the building which receives the RL model after the pre-training.