Data replication and data failover in data storage systems
Replication and failover of data is disclosed. A method includes replicating the data stored in a primary deployment such that the data is further stored in a secondary deployment. The method includes executing one or more transactions on the data at the primary deployment to cause a change to the data to be stored in the primary deployment. The method includes propagating the one or more transactions on the data to the secondary deployment to perform a failover to the secondary deployment while the change to the data is absent from the secondary deployment.
1. A system comprising:
a memory to store data; and
a processor, operatively coupled with the memory, the processor to:
replicate the data stored in a primary deployment such that the data is further stored in a secondary deployment;
execute one or more transactions on the data at the primary deployment to cause a change to the data to be stored in the primary deployment;
determine that the primary deployment transitioned from an available state to an unavailable state;
propagate the one or more transactions on the data to the secondary deployment to perform a failover to the secondary deployment while the change to the data is absent from the secondary deployment; and
adhere to a user-defined maximum acceptable time period for the secondary deployment to become available for executing queries on the data after the primary deployment is determined to be unavailable.
2. The system of claim 1 , wherein the processor is further to shift execution of queries on the data from the primary deployment to the secondary deployment for a duration of time the primary deployment is unavailable.
3. The system of claim 1 , wherein to determine that the primary deployment transitioned from the available state to the unavailable state, the processor to determine one or more of:
a power outage has occurred at the primary deployment;
an error resulting in improper modification or deletion of the data at the primary deployment has occurred;
a data center outage has occurred at the primary deployment;
a cloud provider of the primary deployment has experienced an outage;
an error has occurred at the primary deployment; or
the primary deployment is undergoing scheduled downtime.
4. The system of claim 1 , wherein the processor further to adhere to a user-defined maximum number of database transactions an application may tolerate losing when shifting database operations from the primary deployment to the secondary deployment in response to the primary deployment becoming unavailable.
5. The system of claim 1 , wherein to replicate the data stored in the primary deployment, the processor to replicate in response to the primary deployment becoming unavailable.
6. The system of claim 1 , wherein the processor further to shift a client account connection from the primary deployment to the secondary deployment in response to the primary deployment becoming unavailable.
7. The system of claim 1 , wherein to propagate the one or more transactions to the secondary deployment, the processor to propagate only the one or more transactions without replicating any data already existing in the primary deployment before the primary deployment became unavailable.
8. The system of claim 1 , wherein to propagate the one or more transactions to the secondary deployment, the processor to determine the one or more transactions based on a global file identifier indicating which files in the data have been updated since the primary deployment became unavailable.
9. The system of claim 1 , wherein the primary deployment and the secondary deployment are located in different geographic locations.
10. The system of claim 1 , wherein the primary deployment and the secondary deployment are provided by different cloud-based storage providers.
11. The system of claim 1 , wherein the processor is further to provide a notification to an account associated with the data when availability status of either of the primary deployment or the secondary deployment has changed.
12. A method comprising:
replicating data stored in a primary deployment such that the data is further stored in a secondary deployment;
executing, by a processor, one or more transactions on the data at the primary deployment to cause a change to the data to be stored in the primary deployment;
determining that the primary deployment transitioned from an available state to an unavailable state;
propagating the one or more transactions on the data to the secondary deployment to perform a failover to the secondary deployment while the change to the data is absent from the secondary deployment; and
adhering to a user-defined maximum acceptable time period for the secondary deployment to become available for executing queries on the data after the primary deployment is determined to be unavailable.
13. The method of claim 12 , further comprising, in response to determining that the primary deployment is unavailable, shifting execution of queries on the data from the primary deployment to the secondary deployment for a duration of time the primary deployment is unavailable.
14. The method of claim 13 , wherein shifting the execution of the queries from the primary deployment to the secondary deployment occurs within a user-defined maximum acceptable time period for the secondary deployment to become available for executing queries after the primary deployment is determined to be unavailable.
15. The method of claim 12 , wherein determining that the primary deployment transitioned from the available state to the unavailable state comprises at least one or more of:
a power outage has occurred at the primary deployment;
an error resulting in improper modification or deletion of the data at the primary deployment has occurred;
a data center outage has occurred at the primary deployment;
a cloud provider of the primary deployment has experienced an outage;
an error has occurred at the primary deployment; or
the primary deployment is undergoing scheduled downtime.
16. The method of claim 12 , further comprising:
shifting a client account connection from the primary deployment to the secondary deployment in response to the primary deployment becoming unavailable.
17. The method of claim 12 , wherein propagating the one or more transactions to the secondary deployment comprises propagating only the one or more transactions without replicating any data already existing in the primary deployment before the primary deployment became unavailable.
18. The method of claim 12 , wherein propagating the one or more transactions to the secondary deployment comprises determining the one or more transactions based on a global file identifier indicating which files in the data have been updated since the primary deployment became unavailable.
19. A non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to:
replicate data stored in a primary deployment such that the data is further stored in a secondary deployment;
execute, by the processor, one or more transactions on the data at the primary deployment to cause a change to the data to be stored in the primary deployment;
determine that the primary deployment transitioned from an available state to an unavailable state;
propagate the one or more transactions on the data to the secondary deployment to perform a failover to the secondary deployment while the change to the data is absent from the secondary deployment; and
adhere to a user-defined maximum acceptable time period for the secondary deployment to become available for executing queries on the data after the primary deployment is determined to be unavailable.
20. The non-transitory computer readable storage media of claim 19 , wherein the processor further to, in response to determining that the primary deployment is unavailable, shift execution of queries on the data from the primary deployment to the secondary deployment for a duration of time the primary deployment is unavailable.
21. The non-transitory computer readable storage media of claim 19 , wherein to determine that the primary deployment transitioned from the available state to the unavailable state, the processor is further to at one or more of:
a power outage has occurred at the primary deployment;
an error resulting in improper modification or deletion of the data at the primary deployment has occurred;
a data center outage has occurred at the primary deployment;
a cloud provider of the primary deployment has experienced an outage;
an error has occurred at the primary deployment; or
the primary deployment is undergoing scheduled downtime.
22. The non-transitory computer readable storage media of claim 19 , wherein to replicate the data stored in the primary deployment, the processor to replicate in response to the primary deployment becoming unavailable.
23. The non-transitory computer readable storage media of claim 19 , wherein the processor further to shift a client account connection from the primary deployment to the secondary deployment in response to the primary deployment becoming unavailable.
24. The non-transitory computer readable storage media of claim 19 , wherein to propagate the one or more transactions to the secondary deployment, the processor to propagate only the one or more transactions without replicating any data already existing in the primary deployment before the primary deployment became unavailable.