Allocating background workflows in a data storage system using historical data
View Patent ↗Technologies are described for implementing a system that schedules background workflows in a storage system. The background workflows may be automated maintenance tasks that require data movement, or any other tasks that would otherwise compete with normal input/output operations. Historical load statistics can be analyzed to estimate the preferred times in the future during which to schedule background workflows. After collecting data access statistics and usage patterns for several days or weeks, a number of averages can be calculated from the statistics at various natural periods of system utilization. For example, daily, weekly, or other periods. A weighted average can be established where weighting can place more emphasis on recent data. Such averages can serve as predictors of future loads having similar periodic relationships to the historical load data. Predicting future loads can improve the scheduling of background operations by avoiding conflicts with normal system operations.
1. A method for allocating background workflows in a storage system, the method comprising:
maintaining a history of load on the storage system;
determining a peak load from the history;
determining an expected load by averaging periodic samples of the history, wherein the periodic samples are weighted in the expected load determination to give greater weight to more recently collected periodic samples; and
calculating a permitted background load by subtracting the expected load from the peak load.
2. The method of claim 1 , wherein the periodic samples are from a same time of day for a series of days.
3. The method of claim 1 , wherein the periodic samples are from a series of hours within a same weekday.
4. The method of claim 1 , wherein determining an expected load by averaging periodic samples of the history comprises averaging an hourly average of the history and a weekly weighted average of the history.
5. The method of claim 1 , wherein determining a peak load from the history comprises indentifying a local maxima in a recent subset of the history.
6. The method of claim 1 , further comprising translating the permitted background load to a number of data migrations for a specified time period.
7. A data storage system comprising:
a storage subsystem;
a plurality of physical storage devices associated with the storage subsystem; and
one or more workflow modules operable to cause the storage subsystem to:
maintain a history of load on a storage system;
determine a peak load from the history;
determine an expected load by averaging periodic samples of the history, wherein the periodic samples are weighted in the expected load determination to give greater weight to more recently collected periodic samples; and
calculate a permitted background load by subtracting the expected load from the peak load.
8. The data storage system of claim 7 , wherein the periodic samples are from a same time of day for a series of days.
9. The data storage system of claim 7 , wherein the periodic samples are from a series of hours within a same weekday.
10. The data storage system of claim 7 , wherein determining an expected load by averaging periodic samples of the history comprises averaging an hourly average of the history and a weekly weighted average of the history.
11. The data storage system of claim 7 , wherein determining a peak load from the history comprises indentifying a local maxima in a recent subset of the history.
12. A computer storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to:
maintain a history of load on a storage system;
determine a peak load from the history;
determine an expected load by averaging periodic samples of the history, wherein the periodic samples are weighted in the expected load determination to give greater weight to more recently collected periodic samples; and
calculate a permitted background load by subtracting the expected load from the peak load.
13. The computer storage medium of claim 12 , wherein the periodic samples are from a same time of day for a series of days.
14. The computer storage medium of claim 12 , wherein the periodic samples are from a series of hours within a same weekday.
15. The computer storage medium of claim 12 , wherein determining an expected load by averaging periodic samples of the history comprises averaging an hourly average of the history and a weekly weighted average of the history.
16. The computer storage medium of claim 12 , wherein determining a peak load from the history comprises indentifying a local maxima in a recent subset of the history.
17. The computer storage medium of claim 12 , further causing the computer system to translate the permitted background load to a number of data migrations for a specified time period.