Predicting usage for automated storage tiering
Managing data includes providing at least one logical device that maps sections of the logical device to sections of at least two physical storage areas, where the at least two physical storage areas have different physical storage properties, evaluating characteristics of data associated with at least one of the sections of the logical device, where at least some of the characteristics depend upon predictions as a function of time of future data usage based on past data usage and on heuristic values, and moving the at least one of the sections of the logical device between the at least two storage areas from a first location to a second location according to the characteristics of the data. Each of the at least two physical storage areas may correspond to a different storage tier. Predictions of future data usage may be provided by a state space model.
1. A method for managing data in a data storage system, comprising:
providing at least one logical device that maps sections of the logical device to sections of at least two physical storage areas, wherein the at least two physical storage areas have different physical storage properties;
evaluating historical activity level characteristics of data associated with at least one of the sections of the logical device to determine heuristic values for a state space model for predicting future data activity level;
determining the state space model for predictions of future data activity level, wherein the state space model takes into account a portion of an activity level that changes periodically, trending data and a damping factor for the trending data;
obtaining an estimate of future activity using the state space model and setting a future error to zero, wherein the estimate of future activity is a sum of an estimate of the activity level that changes periodically and an estimate of activity level without a periodic component;
creating a plurality of bins, each bin representing a range of the historical activity level characteristics;
logically grouping a plurality of the sections of the at least one logical device into the plurality of bins based on activity levels of the sections;
moving the at least one of the sections of the logical device between the at least two physical storage areas from a first location to a second location according to at least one of: the bins, and the estimate of future activity using the state space model; and
determining whether to move the at least one of the sections of the logical device based at least in part on the bins for a number of iterations over time, wherein the estimate of future activity is obtained and used as part of determining whether to move the at least one of the sections of the logical device for only a subset of the number of iterations.
2. A method, according to claim 1 , wherein each of the at least two physical storage areas corresponds to a different storage tier.
3. A method, according to claim 1 , wherein the heuristic values are estimated based on a probability distribution of random errors.
4. A method, according to claim 1 , wherein predictions of future data activity level include determining an error value for the state space model.
5. A method, according to claim 1 , further comprising;
using the state space model to determine an estimate of future data activity level based on historical activity level characteristics of the data and a periodic change component.
6. A method, according to claim 1 , further comprising using the state space model to obtain a separate prediction for each hour of a day.
7. A method, according to claim 1 , further comprising using the state space model to obtain a separate prediction for each of a plurality of data objects, wherein each of the plurality of data objects corresponds to one of: a track, a sector, a file, a volume and a relational database object.
8. A method according to claim 1 , wherein each bin is defined to have a different range of activity level, and each bin includes one or more sections of the at least one logic device determined to have an activity level within the defined range of activity level of the bin.
9. Computer software, provided in a non-transitory computer-readable medium, that manages data provided in at least one logical device of a data storage system that maps sections of the at least one logical device to sections of at least two physical storage areas that have different physical storage properties, the software comprising:
executable code that evaluates historical activity level characteristics of data associated with at least one of the sections of the logical device to determine heuristic values for a state space model for predicting future data activity level;
executable code that determines the state space model for predictions of future data activity level, wherein the state space model takes into account a portion of an activity level that changes periodically, trending data and a damping factor for the trending data;
executable code that obtains an estimate of future activity using the state space model and that sets a future error to zero, wherein the estimate of future activity is a sum of an estimate of the activity level that changes periodically and an estimate of activity level without a periodic component;
executable code that creates a plurality of bins, each bin representing a range of the historical activity level characteristics;
executable code that logically groups a plurality of the sections of the at least one logical device into the plurality of bins based on activity levels of the sections;
executable code that moves the at least one of the sections of the logical device between the at least two physical storage areas from a first location to a second location according to at least one of: the bins, and the estimate of future activity using the state space model; and
executable code that determines whether to move the at least one of the sections of the logical device based at least in part on the bins for a number of iterations over time, wherein the estimate of future activity is obtained and used as part of determining whether to move the at least one of the sections of the logical device for only a subset of the number of iterations.
10. Computer software, according to claim 9 , wherein each of the at least two physical storage areas corresponds to a different storage tier.
11. Computer software, according to claim 9 , wherein the heuristic values are estimated based on a probability distribution of random errors.
12. Computer software, according to claim 9 , wherein predictions of future data activity level include determining an error value for the state space model.
13. Computer software, according to claim 9 , further comprising:
using the state space model to determine an estimate of future data activity level based on historical activity level characteristics of the data and a periodic change component.
14. Computer software, according to claim 9 , further comprising using the state space model to obtain a separate prediction for each hour of a day.
15. Computer software, according to claim 9 , further comprising using the state space model to obtain a separate prediction for each of a plurality of data objects, wherein each of the objects corresponds to one of: a track, a sector, a file, a volume and a relational database object.
16. Computer software, according to claim 9 , wherein each bin is defined to have a different range of activity level, and each bin includes one or more sections of the at least one logic device determined to have an activity level within the defined range of activity level of the bin.