Methods, systems and apparatus to dynamically facilitate boundaryless, high availability system management
In a Boundaryless Control High Availability (“BCHA”) system (e.g., industrial control system) comprising multiple computing resources (or computational engines) running on multiple machines, technology for computing in real time the overall system availability based upon the capabilities/characteristics of the available computing resources, applications to execute and the distribution of the applications across those resources is disclosed. In some embodiments, the disclosed technology can dynamically manage, coordinate recommend certain actions to system operators to maintain availability of the overall system at a desired level. High Availability features may be implemented across a variety of different computing resources distributed across various aspects of a BCHA system and/or computing resources. Two example implementations of BCHA systems described involve an M:N working configuration and M:N+R working configuration.
1 . A system for dynamically load-balancing redistribution elements across a group of computing resources that facilitate at least an aspect of an Industrial Execution Process, comprising:
a system configured to:
monitor the operational state data and availability metrics associated with an Execution Process control system components/computing resources;
simulate computing resource failures iteratively for each of the Execution Process control system component/computing resource;
determine an aggregated operational state for HA−M:N Validation load-balancing based on the simulation wherein aggregated operational state is used to determine a transition supplemental resource request to facilitate transitioning operational state back to a viable M:N working configuration and maintain high availability requirements;
identify a load-balancing opportunity to trigger redistribution of at least one redistribution element to a proposed redistribution target wherein the load-balancing opportunity involves a Resource Failure Detection Load-Balancing and a HA−M:N Validation which executes a Z Validation Degree computing resource failure simulation, wherein Z is greater than or equal to 1;
process the Execution Process control system components/computing resources to determine a target redistribution pool of potential redistribution candidates that maintain system availability constraints;
generate an operational state transition Supplement computing resource request to facilitate transitioning back to a viable M:N working configuration that maintains high availability requirements; and
redistribute the at least one redistribution element to at least one target redistribution component/computing resource selected from the target redistribution pool.
2 . The system of claim 1 , further comprising:
selecting the redistribution target from the target redistribution pool according to a Load-Balancing Redeployment Type.
3 . The system of claim 2 wherein the Load-Balancing Redeployment Type includes one of: Fastest Failover Load Balancing, Optimal Failure Load Balancing, or Balanced Failure Load Balancing.
4 . The system of claim 1 , further comprising:
redistributing one or more Applications as the at least one redistribution element.
5 . The system of claim 1 , further comprising:
redistributing one or more Applications and corresponding Work Items as the at least one redistribution element.
6 . The system of claim 5 , wherein the at least one redistribution element is associated with a failed computing resource.
7 . The system of claim 1 , further comprising:
executing a remediation operational state determination.
8 . The system of claim 7 wherein the remediation operational state determination validates that computing resources are facilitating a viable M:N working configuration and maintain high availability requirements.
9 . The system of claim 7 wherein the remediation operational state determination validates that computing resources are facilitating a viable M:N working configuration and does not maintain high availability requirements.
10 . The system of claim 7 wherein the remediation operational state determination validates that computing resources are not facilitating a viable M:N working configuration and does not maintain high availability requirements.
11 . The system of claim 1 , further comprising:
executing auto-remediation to obtain and provision the Supplement computing resource; and
redistributing at least one redistribution element to the Supplement computing resource.
12 . The system of claim 5 , wherein the Work Items include:
executing auto-remediation to obtain and provision the Supplement computing resource; and
redistributing at least one redistribution element to the Supplement computing resource.
13 . The system of claim 5 , further comprising:
activating redistributed elements along with corresponding application or work item operational data stored in a data services module at or near the time of the computing resource failure.
14 . A method for dynamically load-balancing redistribution elements across a group of computing resources that facilitate at least an aspect of an Industrial Execution Process comprising:
monitoring operational state data and availability metrics associated with Execution Process control system components/computing resources;
simulating computing resource failures iteratively for each of the Execution Process control system component/computing resource;
determining an aggregated operational state for HA−M:N Validation load-balancing based on the simulation wherein aggregated operational state is used to determine a transition supplemental resource request to facilitate transitioning operational state back to a viable M:N working configuration and maintain high availability requirements;
identifying a load-balancing opportunity to trigger redistribution of at least one redistribution element to a proposed redistribution target wherein the load-balancing opportunity involves a Resource Failure Detection Load-Balancing and a HA−M:N Validation which executes a Z Validation Degree computing resource failure simulation, wherein Z is greater than or equal to 1;
processing Execution Process control system components/computing resources to determine a target redistribution pool of potential redistribution candidates that maintain system availability constraints;
generate an operational state transition Supplement computing resource request to facilitate transitioning back to a viable M:N working configuration that maintains high availability requirements; and
redistributing the at least one redistribution element to at least one target redistribution component/computing resource selected from the target redistribution pool.
15 . The method of claim 14 , further comprising:
selecting the redistribution target from the target redistribution pool according to a Load-Balancing Redeployment Type.
16 . The method of claim 15 wherein the Load-Balancing Redeployment Type includes one of: Fastest Failover Load Balancing, Optimal Failure Load Balancing, or Balanced Failure Load Balancing.
17 . The method of claim 14 , further comprising:
redistributing one or more Applications as the at least one redistribution element.
18 . The method of claim 14 , further comprising:
redistributing one or more Applications and corresponding Work Items as the at least one redistribution element.
19 . The method of claim 18 , wherein the at least one redistribution element is associated with a failed computing resource.
20 . The method of claim 14 , further comprising:
executing a remediation operational state determination.
21 . The method of claim 20 wherein the remediation operational state determination validates that computing resources are facilitating a viable M:N working configuration and maintain high availability requirements.
22 . The method of claim 20 wherein the remediation operational state determination validates that computing resources are facilitating a viable M:N working configuration and does not maintain high availability requirements.
23 . The method of claim 20 wherein the remediation operational state determination validates that computing resources are not facilitating a viable M:N working configuration and does not maintain high availability requirements.
24 . The method of claim 14 , further comprising:
executing auto-remediation to obtain and provision the Supplement computing resource; and
redistributing at least one redistribution element to the Supplement computing resource.
25 . The method of claim 18 , wherein the Work Items include:
a computational or control component generated based on a Concrete Process Control Model and associated computational or control component element of the Execution Process; and
an abstraction model component generated based on operational characteristics and capabilities associated with the work item.
26 . The method of claim 18 , further comprising:
activating redistributed elements along with corresponding application or work item operational data stored in a data services module at or near the time of the computing resource failure.
27 . A non-transitory computer readable medium storing sequences of computer-executable instructions for dynamically load-balancing redistribution elements across a group of computing resources that facilitate at least an aspect of an Industrial Execution Process, the sequences of computer-executable instructions including instructions that instruct at least one processor to:
monitor the operational state data and availability metrics associated with an Execution Process control system components/computing resources;
simulate computing resource failures iteratively for each of the Execution Process control system component/computing resource;
determine an aggregated operational state for HA−M:N Validation load-balancing based on the simulation wherein aggregated operational state is used to determine a transition supplemental resource request to facilitate transitioning operational state back to a viable M:N working configuration and maintain high availability requirements;
identify a load-balancing opportunity to trigger redistribution of at least one redistribution element to a proposed redistribution target wherein the load-balancing opportunity involves a Resource Failure Detection Load-Balancing and a HA−M:N Validation which executes a Z Validation Degree computing resource failure simulation, wherein Z is greater than or equal to 1;
process the Execution Process control system components/computing resources to determine a target redistribution pool of potential redistribution candidates that maintain system availability constraints;
generate an operational state transition Supplement computing resource request to facilitate transitioning back to a viable M:N working configuration that maintains high availability requirements; and
redistribute the at least one redistribution element to at least one target redistribution component/computing resource selected from the target redistribution pool.