Systems and methods for fault recovery in multi-tier applications
A computer-implemented method for fault recovery in multi-tier applications may include: 1) identifying a plurality of clusters, 2) identifying a multi-tier application that includes a plurality of components, each cluster within the plurality of clusters hosting a component, 3) identifying a fault of a first component within the plurality of components on a first cluster within the plurality of clusters, the fault requiring a first recovery action, 4) identifying at least one dependency relationship involving the first component and a second component within the plurality of components on a second cluster within the plurality of clusters, 5) determining, based on the fault and the dependency relationship, that the second component requires a second recovery action to ensure that the multi-tier application operates correctly, and 6) performing the second recovery action on the second component. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for fault recovery in multi-tier applications, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
identifying a plurality of clusters, wherein a first cluster and a second cluster within the plurality of clusters do not share availability information with each other;
identifying a multi-tier application comprising a plurality of components, each cluster within the plurality of clusters hosting a component within the plurality of components;
identifying a fault of a first component within the plurality of components on the first cluster within the plurality of clusters, the fault requiring a first recovery action to ensure that the multi-tier application operates correctly, wherein identifying the fault comprises probing the first component with an attempted communication and determining the fault based on the outcome of the attempted communication;
identifying at least one dependency relationship involving the first component and a second component within the plurality of components on the second cluster within the plurality of clusters;
determining, based on the fault and the dependency relationship, that the second component requires a second recovery action to ensure that the multi-tier application operates correctly;
performing the second recovery action on the second component.
2. The computer-implemented method of claim 1 , further comprising:
determining, based on the dependency relationship, an order in which to perform the first recovery action and the second recovery action;
performing the first recovery action and the second recovery action in the determined order.
3. The computer-implemented method of claim 1 , wherein the first recovery action comprises restarting the first component.
4. The computer-implemented method of claim 1 , wherein the second recovery action comprises restarting the second component.
5. The computer-implemented method of claim 1 , wherein each cluster within the plurality of clusters comprises a high-availability cluster.
6. The computer-implemented method of claim 5 , wherein identifying the fault comprises receiving availability information from the first cluster.
7. The computer-implemented method of claim 1 , wherein the plurality of clusters comprises a plurality of heterogeneous clustering platforms.
8. The computer-implemented method of claim 1 , wherein each cluster within the plurality of clusters comprises a cross-cluster availability agent for determining dependencies within the plurality of components.
9. A system for fault recovery in multi-tier applications, the system comprising:
an identification module programmed to:
identify a plurality of clusters, wherein a first cluster and a second cluster within the plurality of clusters do not share availability information with each other;
identify a multi-tier application comprising a plurality of components, each cluster within the plurality of clusters hosting a component within the plurality of components;
a monitoring module programmed to identify a fault of a first component within the plurality of components on the first cluster within the plurality of clusters, the fault requiring a first recovery action to ensure that the multi-tier application operates correctly, wherein identifying the fault comprises probing the first component with an attempted communication and determining the fault based on the outcome of the attempted communication;
a dependency module programmed to:
identify at least one dependency relationship involving the first component and a second component within the plurality of components on the second cluster within the plurality of clusters;
determine, based on the fault and the dependency relationship, that the second component requires a second recovery action to ensure that the multi-tier application operates correctly;
a recovery module programmed to perform the second recovery action on the second component;
at least one processor configured to execute the identification module, the monitoring module, the dependency module, and the recovery module.
10. The system of claim 9 , wherein:
the dependency module is further programmed to determine, based on the dependency relationship, an order in which to perform the first recovery action and the second recovery action;
the recovery module is further programmed to perform the first recovery action and the second recovery action in the determined order.
11. The system of claim 9 , wherein the first recovery action comprises restarting the first component.
12. The system of claim 9 , wherein the second recovery action comprises restarting the second component.
13. The system of claim 9 , wherein each cluster within the plurality of clusters comprises a high-availability cluster.
14. The system of claim 13 , wherein the monitoring module is programmed to identify the fault by receiving availability information from the first cluster.
15. The system of claim 9 , wherein the plurality of clusters comprises a plurality of heterogeneous clustering platforms.
16. The system of claim 9 , wherein each cluster within the plurality of clusters comprises a cross-cluster availability agent for determining dependencies within the plurality of components.
17. A computer-readable-storage medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
identify a plurality of clusters, wherein a first cluster and a second cluster within the plurality of clusters do not share availability information with each other;
identify a multi-tier application comprising a plurality of components, each cluster within the plurality of clusters hosting a component within the plurality of components;
identify a fault of a first component within the plurality of components on the first cluster within the plurality of clusters, the fault requiring a first recovery action to ensure that the multi-tier application operates correctly, wherein identifying the fault comprises probing the first component with an attempted communication and determining the fault based on the outcome of the attempted communication;
identify at least one dependency relationship involving the first component and a second component within the plurality of components on the second cluster within the plurality of clusters;
determine, based on the fault and the dependency relationship, that the second component requires a second recovery action to ensure that the multi-tier application operates correctly;
perform the second recovery action on the second component.
18. The computer-readable medium of claim 17 , wherein the one or more computer-executable instructions further cause the computing device to:
determine, based on the dependency relationship, an order in which to perform the first recovery action and the second recovery action;
perform the first recovery action and the second recovery action in the determined order.
19. The computer-readable medium of claim 17 , wherein the first recovery action comprises restarting the first component.
20. The computer-readable medium of claim 17 , wherein the second recovery action comprises restarting the second component.