Dynamic snapshot scheduling for data storage
Techniques are provided for dynamic snapshot scheduling. In an example, a dynamic snapshot scheduler can analyze historical data about storage system resources. The dynamic snapshot scheduler can use this historical data to predict how the storage system resources will be used in the future. Based on this prediction, the dynamic snapshot scheduler can schedule snapshot activities for one or more times that are relatively unlikely to experience system resource contention. The dynamic snapshot scheduler can then initiate snapshot activities at those scheduled times.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
predicting future performance data associated with the system based on historical performance data associated with the system;
dividing the future performance data into time chunks;
determining statistical data about the time chunks;
determining that a first time chunk and a second time chunk of the time chunks are similar based on applying a dynamic time warping technique to the statistical data according to a similarity criterion;
selecting the first time chunk based on a first likelihood of resource contention at a first time indicated by the first time chunk being determined to be lower than a second likelihood of resource contention at a second time indicated by the second time chunk; and
performing a snapshot operation to generate a snapshot of the system at the first time.
2. The system of claim 1 , wherein the historical performance data comprises data representative of processing resources consumed on non-snapshot operations other than the snapshot operation.
3. The system of claim 1 , wherein the operations further comprise:
determining a first size of the first time chunk based on a result of adjusting a hyperparameter for the first time chunk through multiple iterations.
4. The system of claim 1 , wherein the operations further comprise:
determining a first size of the first time chunk based on how much time a previous snapshot has taken to complete.
5. The system of claim 1 , wherein each time chunk of the time chunks has a same size.
6. The system of claim 1 , wherein the statistical data comprises at least one of mean data representative of a mean determined from the time chunks and standard deviation data representative of a standard deviation determined from the time chunks.
7. A method, comprising:
predicting, by a system comprising a processor, future performance data of the system based on historical performance data of the system;
dividing the future performance data into time chunks;
identifying a subset of the time chunks based on similarity values of the time chunks, the similarity values being determined based on applying a dynamic time warping technique to the future performance data;
selecting a first time chunk of the subset of time chunks based on a first likelihood of resource contention at a first time indicated by the first time chunk being lower than a second likelihood of resource contention at a second time indicated by a second time chunk of the subset of time chunks; and
generating a snapshot of the system at the first time.
8. The method of claim 7 , wherein performing the selecting of the first time chunk is based on determining that the first time chunk has a smaller mean than the second time chunk.
9. The method of claim 7 , wherein the selecting of the first time chunk is based on determining that the first time chunk has a smaller standard deviation than the second time chunk.
10. The method of claim 7 , wherein the selecting of the first time chunk comprises:
selecting a third time chunk of the subset of time chunks.
11. The method of claim 10 , wherein the generating of the snapshot comprises:
generating the snapshot at the first time and generating another snapshot at a third time.
12. The method of claim 7 , further comprising:
removing anomalous data from the historical performance data before the predicting of the future performance data of the system based on the historical performance data of the system.
13. The method of claim 12 , wherein the anomalous data comprises:
data that falls outside of a statistical threshold for an upper bound or a lower bound for less than a first threshold amount of time.
14. A non-transitory computer-readable medium comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising:
dividing predicted future performance data associated with a predicted future performance of the system into a group of time chunks;
identifying a subgroup of the group of time chunks based on similarity values of the group of time chunks, the similarity values being determined based on applying dynamic time warping to the predicted future performance data;
selecting a first time chunk of the subgroup of group of time chunks based on a first likelihood of resource contention at a first time indicated by the first time chunk being lower than a second likelihood of resource contention at a second time indicated by a second time chunk of the subgroup of group of time chunks; and
performing a snapshot of the system at the first time.
15. The non-transitory computer-readable medium of claim 14 , wherein the selecting of the first time chunk is based on determining that the first time chunk has a higher similarity value than the second time chunk.
16. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprise:
determining the predicted future performance data of the system based on previous performance data associated with a previous performance of the system.
17. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprise:
determining the predicted future performance data of the system based on previous configuration data previously applied to the system.
18. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprise:
determining the predicted future performance data of the system based on previous snapshot data representative of a previous snapshot of the system.
19. The non-transitory computer-readable medium of claim 18 , wherein the previous snapshot data comprises at least one of:
schedule data representative of a schedule for snapshot activity, frequency data representative of a frequency of snapshot activity, first time data representative of a start time of snapshot activity, second time data representative of an end time of snapshot activity, and third time data representative of a total run time of snapshot activity.
20. The non-transitory computer-readable medium of claim 14 , wherein the statistical data comprises at least one of mean data representative of a mean determined from the time chunks and standard deviation data representative of a standard deviation determined from the time chunks.