Method to optimize restore based on data protection workload prediction
An intelligent method of selecting a data recovery site upon receiving a data recovery request. The backup system collects historical activity data of the storage system to identify work load of every data recovery site. A predicted activity load for each data recovery site is then generated using the collected data. When a request for data recovery is received, the system first identifies which data recovery site has copies of the files to be recovered. Then it uses the predicted work load for these data recovery sites to determine whether to use a geographically local site or a site that may be remote geographically, but has a lower work load.
1. In a backup system comprising a backup manager and a plurality of data recovery (DR) sites, each DR site storing backup files, a method of directing restore requests, the method comprising:
collecting backup activity data of each site for a period of time including read, write, replication, stream, and usage of CPU and I/O load on every replication set from each of the DR sites to generate a prediction of future activity load for each DR site by using K-means method and a machine learning (ML) by splitting a total data set into a test training set and a test testing set with a ratio of 7:3;
generating the prediction of future activity load for each DR site by using the activity data to predict future activity load for each site, wherein the prediction is to predict workload status for each site for a set period time;
receiving a restore request;
determining which of the sites stores required backup files by using the future activity load based on the prediction to determine which of the sites has low activity load when no preferred DR site is selected by a user; and
forwarding the restore request to the site having the low activity load.
2. The method of claim 1 , wherein collecting backup activity data comprises collecting I/O and CPU activity.
3. The method of claim 1 , further comprising quantizing the future activity load into N activity levels.
4. The method of claim 3 , wherein the N activity levels include three levels: busy, average, and idle.
5. The method of claim 1 , further comprising storing in a file system each address of each replica of each file with a corresponding indication of predicted activity load level for the site corresponding to the address.
6. The method of claim 1 , further comprising upon receiving a restore request:
designating one of the sites as local site;
determining whether additional sites store replicas of files stored in the local site; and
when it is determined that no other site stores a replica, forwarding the request to the local site.
7. The method of claim 6 , further comprising:
when it is determined that other site stores a replica, determining whether the other site's predicted activity level is lower than predicted activity level of the local site and, if so, forwarding the request to the other site.
8. The method of claim 1 , wherein collecting backup activity data is performed periodically and is used to predict future activity load.
9. A system comprising:
a plurality of storage systems distributed in a plurality of data recovery (DR) sites;
an analytical engine;
a backup manager;
a load balancer;
wherein the analytical engine, backup manager, and load balancer employ one or more processors; and
a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to perform the steps:
collect backup activity data of each of the DR sites for a period of time including read, write, replication, stream, and usage of CPU and I/O load on every replication set from each of the DR sites to generate a prediction of future activity load for each DR site by using K-means method and a machine learning (ML) by splitting a total data set into a test training set and a test testing set with a ratio of 7:3;
generate the prediction of future activity load for each DR site by using the activity data to predict future activity load for each of the DR sites, wherein the prediction is to predict workload status for each site for a set period time;
upon receiving a restore request, determine which of the DR sites stores required backup files by using the future activity load based on the prediction to determine which of the sites has low activity load when no preferred DR site is selected by a user; and
forward the restore request to the DR site having the low activity load.
10. The system of claim 9 , wherein the collect backup activity data step is performed periodically and is used to update the predicted future activity load.
11. The system of claim 10 , wherein the collect backup activity data step comprises collecting I/O and CPU activity from each of the DR site.
12. The system of claim 11 , further comprising the step further comprising quantizing the future activity load into N activity levels.
13. The system of claim 12 , wherein the N activity levels include three levels: busy, average, and idle.
14. The system of claim 11 , further comprising the step of storing in a file system each address of each replica of each file with a corresponding indication of predicted activity load level for the site corresponding to the address.
15. The system of claim 14 , further comprising the step, upon receiving a restore request:
designating one of the sites as local site;
determining whether additional sites store replicas of files stored in the local site; and
when it is determined that no other site stores a replica, forwarding the request to the local site.
16. A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code operating in a computing cluster and including instructions to:
collect backup activity data of each of the DR sites for a period of time including read, write, replication, stream, and usage of CPU and I/O load on every replication set from each of the DR sites to generate a prediction of future activity load for each DR site by using K-means method and a machine learning (ML) by splitting a total data set into a test training set and a test testing set with a ratio of 7:3;
generate the prediction of future activity load for each DR site by using the activity data to predict future activity load for each of the DR sites, wherein the prediction is to predict workload status for each site for a set period time;
upon receiving a restore request, determine which of the DR sites stores required backup files by using the future activity load based on the prediction to determine which of the sites has low activity load when no preferred DR site is selected by a user; and
forward the restore request to the DR site having the low activity load.
17. The computer program product of claim 16 , wherein the program code includes further instructions to store in a file system each address of each replica of each file with a corresponding indication of predicted activity load level for the site corresponding to the address.
18. The computer program product of claim 17 , wherein the program code includes further instructions to periodically collect the backup activity data and update the predicted future activity load.
19. The computer program product of claim 18 , upon receiving a restore request:
designating one of the sites as local site;
determining whether additional sites store replicas of files stored in the local site; and
when it is determined that no other site stores a replica, forwarding the request to the local site.
20. The computer program product of claim 19 , wherein collecting backup activity data step comprises collecting I/O and CPU activity from each of the DR site.