Method and system for rollback-free failure recovery of multi-step procedures
View Patent ↗Disclosed is a rollback-free method for performing multi-step procedures in the presence of possible failure. As the procedure proceeds from its initial state through transitions to its final state, its constituent transitions are monitored for failure. If a failure is detected, then the procedure is “sidetracked” into a recovery coordination state. From the recovery coordination state, the procedure is logically taken back to its initial state and retried. In this manner, the procedure is shepherded, without rollbacks, through its transitions until it successfully reaches its final state. In a particular embodiment, a multi-step procedure is developed for moving a resource from one resource server to another The steps of the procedure are designed so that all throughout the procedure, both of the resource servers and a directory server are kept synchronized. This allows client requests to proceed without interruption even during the resource movement.
1. A method in a computing system for recovering from a failure when transferring a resource from a first server to a second server, the method comprising:
indicating that the resource is to be transferred from the first server to the second server;
starting at an initial step execution of a multi-step procedure that effects the transferring of the resource from the first server to the second server as indicated and updating of a directory server that currently references the first server as repository of the resource to reference the second server as repository of the resource, each step of the procedure setting a state and being idempotent such that re-execution of the step with the set state will have the same result as the initial execution of the step, the procedure including copying the resource from the first server to the second server, deleting the resource from the first server, and setting the directory server to reference the second server;
when an error occurs, restarting at the initial step the execution of the procedure without rolling back state of the previously executed steps so that each step executed after the error occurs will have the same result as when executed before the error occurs; and
when the execution of the procedure completes, indicating that the resource is not being transferred from the first server.
2. The method of claim 1 wherein whenever it is indicated that the resource is to be transferred from the first server to the second server while the procedure is not executing, restarting execution of the procedure.
3. The method of claim 1 wherein the steps of the procedure do not check for state indicating that an error previously occurred.
4. The method of claim 1 wherein the indicating that the resource is to be transferred from the first server to the second server both identifies the second server and indicates that the transfer to the second server has not yet been completed.
5. The method of claim 1 wherein the procedure performs the same steps during execution regardless of whether an error occurred.
6. A method in a computing system for recovering from a failure when transferring a resource from a first server to a second server, the method comprising:
indicating that the resource is to be transferred from the first server to the second server;
starting at an initial step execution of a procedure that effects the transferring of the resource from the first server to the second server as indicated, each step of the procedure setting a state and being idempotent such that re-execution of the step with the set state will have the same effect as the initial execution of the step;
when an error occurs, restarting at the initial step the execution of the procedure without rolling back state of the previously executed steps so that each step executed after the error occurs will have the same effect as when executed before the error occurs; and
when the execution of the procedure completes, indicating that the resource is not being transferred from the first server
wherein a current resource flag indicates the first server and the indicating includes setting a target resource flag to indicate the second server and including:
setting a moveaway flag of the first server to indicate moveaway;
copying the resource from the first server to the second server;
setting a moveaway flag of the second server to indicate not moveaway;
setting the target resource flag to not indicate the second server; and
setting the current resource flag to indicate the second server.
7. The method of claim 6 including after setting the moveaway flag of the second server, deleting the resource from the first server.
8. The method of claim 6 wherein the setting of the current resource flag to indicate the second server and the selling of the target resource flag to not indicate the second server are performed as an atomic operation.
9. A computing system for recovering from a failure when transferring a resource from a first device to a second device, comprising:
a component that sets an indication that the resource is to be transferred from the first device to the second device;
a component that starts at an initial step execution of a procedure that transfers the resource from the first device to the second device as indicated by indication, each step of the procedure selling a state and being idempotent such that re-execution of the step with the set state will have the same effect as the initial execution of the step;
a component that detects an error in transferring the resource and restarts execution of the procedure at the initial step without rolling back state of the previously executed steps so that each step previously executed is either re-executed and has the same effect as when executed before the error occurs or skipped relying on the effect of the previous execution of the step; and
a component that, upon completion of the transfer, sets the indication to indicate that the resource is not being transferred from the first device
wherein a current resource flag is set to the first device and the computing system sets a target resource flag to indicate the second device, sets a moveaway flag of the first device to indicate moveaway, copies the resource from the first device to the second device, sets a moveaway flag of the second device to indicate not moveaway, sets the target resource flag to not indicate the second device; and sets the current resource flag to indicate the second device.
10. The computing system of claim 9 wherein the computing system further deletes the resource from the first device after the moveaway flag of the second device is set.
11. The computing system of claim 9 wherein the selling of the current resource flag to indicate the second device and the selling of the target resource flag to not indicate the second device are performed as an atomic operation.
12. A computer-storage medium encoded within computer-executable instructions for controlling computer system to recovering from failure when transferring a resource from a first server to a second server, by a method comprising:
setting an indication that indicates that the resource is to be transferred from the first server to the second server;
starting at an initial step execution of a procedure that transfers the resource from the first server to the second server as indicated by the indication, each step of the procedure setting a state and being idempotent such that re-execution of the step with the set state will have the same effect as the initial execution of the step;
when an error occurs, restarting at the initial step the execution of the procedure without rolling back state of the previously executed steps in such a way that each step is either skipped or re-executed so that after skipping or re-executing a step after the error occurs will result in the same effect as when executed before the error occurs; and
when the execution of the procedure completes, setting the indication to indicate that the resource is not being transferred further including:
setting a current resource flag to indicate the first server;
setting a target resource flag to indicate the second server;
setting a moveaway flag of the first server to indicate moveaway;
copying the resource from the first server to the second server;
setting a moveaway flag of the second server to indicate not moveaway;
setting the target resource flag to not indicate the second server; and
setting the current resource flag to indicate the second server.
13. The computer storage medium of claim 12 wherein the setting of the current resource flag to indicate the second server and the setting of the target resource flag to not indicate the second server are performed as an atomic operation.
14. The computer storage medium of claim 12 wherein whenever it is detected that the indication is set to indicate that the resource is to be transferred from the first server to the second server while the procedure is not executing, restarting execution of the procedure.