BUILDING SYSTEM WITH BROKERING ARCHITECTURE TO PROVIDE BUMP-LESS FAILOVER AND REPLAY
A distributed control system with bump-less failover and replay capability is disclosed. The distributed control system stores the state of each control application in a separate storage system. Each time the control applications runs it can store the results of the calculations required for a subsequent calculation. When a compute environment running the control applications fails another environment can begin performing the same operations. Replay capability is provided to facilitate the investigation of the control application's execution.
1 . One or more memory devices having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
operating a first isolated compute environment during a first time period to perform a first set of calculations related to control of a building and send results of the first set of calculations;
operating an additional plurality of isolated compute environments to, in combination, perform a second set of calculations related to the control of the building and send results of the second set of calculations;
operating a second isolated compute environment to store the results of the first set of calculations and the results of the second set of calculations; and
instructing a failover set of isolated compute environments from the additional plurality of isolated compute environments to perform the first set of calculations during a second time period in response to the first isolated compute environment encountering a failure when performing the first set of calculations during the first time period.
2 . The one or more memory devices of claim 1 , wherein the second time period comprises a duration of at least one of:
the failure of the first isolated compute environment; or
a restart of the first isolated compute environment.
3 . The one or more memory devices of claim 1 , wherein the first isolated compute environment and the failover set of isolated compute environments store the results of the first set of calculations and the results of the second set of calculations in the second isolated compute environment when performing the first set of calculations and the second set of calculations.
4 . The one or more memory devices of claim 1 , wherein the second isolated compute environment is configured to store calculations inside the second isolated compute environment and inside a database management system, wherein the operations further comprise recreating the second isolated compute environment using the database management system in response to an outage of the second isolated compute environment.
5 . The one or more memory devices of claim 1 , the operations further comprising:
operating the second isolated compute environment to receive measurements from an input-output module;
operating the first isolated compute environment to send commands to the input-output module to apply to the building; and
operating the first isolated compute environment to receive, from the input-output module, a third time period for which it was not applying the commands.
6 . The one or more memory devices of claim 5 , the operations further comprising:
determining a first output of the input-output module at a start of the third time period; and
determining an initial action predicted to produce a second output that satisfies a similarity criterion with the first output when the initial action is performed by the input-output module.
7 . The one or more memory devices of claim 1 , the operations further comprising predicting a fault in the first isolated compute environment.
8 . A method for providing disturbance-free control of a building after failure of a control device, the method comprising:
operating a first isolated compute environment during a first time period to perform a first set of calculations related to control of the building and send results of the first set of calculations;
operating an additional plurality of isolated compute environments to, in combination, perform a second set of calculations related to the control of the building and send results of the second set of calculations;
operating a second isolated compute environment to store the results of the first set of calculations and the results of the second set of calculations; and
instructing a failover set of isolated compute environments from the additional plurality of isolated compute environments to perform the first set of calculations during a second time period in response to the first isolated compute environment encountering a failure when performing the first set of calculations during the first time period.
9 . The method of claim 8 , wherein the second time period comprises a duration of at least one of:
the failure of the first isolated compute environment; or
a restart of the first isolated compute environment.
10 . The method of claim 8 , wherein the first isolated compute environment and the failover set of isolated compute environments store the results of the first set of calculations and the results of the second set of calculations in the second isolated compute environment when performing the first set of calculations and the second set of calculations.
11 . The method of claim 8 , further comprising:
storing calculations inside a database management system; and
recreating data within the second isolated compute environment using the database management system in response to an outage of the second isolated compute environment.
12 . The method of claim 8 , further comprising:
applying outputs to the building with an input-output module; and
determining a start of a third time period during which the input-output module was not applying the outputs.
13 . The method of claim 12 , further comprising:
determining a first output of the input-output module at the start of the third time period; and
determining an initial action predicted to produce a second output that satisfies a similarity criterion with the first output when the initial action is performed by the input-output module.
14 . The method of claim 8 , further comprising predicting a fault in the first isolated compute environment.
15 . One or more memory devices having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
controlling equipment of a building based on measurements received from sensors related to the building by executing a control or monitoring application and sending commands to actuators of the building within one or more isolated compute environments;
creating a second isolated compute environment representing at least a portion of the one or more isolated compute environments using images of the one or more isolated compute environments;
simulating an input-output module, the simulating comprising sending messages containing past measurements to the control or monitoring application operating in the second isolated compute environment when a simulated time of the control or monitoring application operating in the second isolated compute environment matches a time the past measurements occurred in the one or more isolated compute environments; and
executing a calculation of the second isolated compute environment when the simulated time of the control or monitoring application operating in the second isolated compute environment matches a calculation time the calculation was executed in the one or more isolated compute environments.
16 . The one or more memory devices of claim 15 , the operations further comprising executing lines of code within the second isolated compute environment in sequence and showing results of executing the lines of code after an execution.
17 . The one or more memory devices of claim 15 , wherein simulating the input-output module comprises sending the messages containing simulated measurements based on a statistical analysis of the past measurements.
18 . The one or more memory devices of claim 15 , wherein the second isolated compute environment executes the control or monitoring application operating in the second isolated compute environment while the control or monitoring application operating in the one or more isolated compute environments continues to operate on the building.
19 . The one or more memory devices of claim 15 , the operations further comprising comparing an output of the calculation of the second isolated compute environment with an output of the calculation executed in the one or more isolated compute environments and creating an indication of a difference based on the comparison.
20 . The one or more memory devices of claim 15 , wherein a first portion of the one or more isolated compute environments run on a node of a cluster of computers and a second portion of the one or more isolated compute environments run on the input-output module.