Multi-dimensional placement
A multi-dimensional placement methodology, system and computer readable medium is presented. A plurality of data sets is ordered by need. A plurality of storage areas are defined based on a storage device type, an associated compression algorithm, and a plurality of parameters associated with different properties of the particular storage device and the compression algorithm being used. A data set is placed in a selected storage area based on a determination of which storage area provides a desired combination of the storage device type and compression.
1. A method for multi-dimensional determination of data placement in storage devices of a storage system, the method comprising:
ordering a plurality of data sets, wherein data sets identified as having a greater number of reads and writes than others of the data sets are placed higher in the ordering;
defining a plurality of storage areas based on storage device types of the storage devices, associated compression algorithms, and a plurality of parameters associated with different properties of the storage devices and the compression algorithms being used;
storing a first data set of the plurality of data sets in a selected storage area of one of the storage devices based on a determination of which of the storage areas provides a desired combination of compression and storage device type for the first data set; and
storing, in another selected storage area of one of the storage devices, a next data set of the plurality of data sets based on the ordering, the storing the next data set is based on a determination of which of the storage areas provides a desired combination of compression and storage device type for the next data set;
wherein utilization of resources of the storage system is a function of the desired combination, and the desired combination is configured to prevent overutilization of the resources.
2. The method of claim 1 wherein ordering a plurality of data sets further comprises ordering the data sets according to a service level objective (SLO) associated with each of the plurality of data sets, a number of writes associated with each of the plurality of data sets, and a number of reads associated with each of the plurality of data sets.
3. The method of claim 1 wherein the selected storage area is determined according to a formula comprising:
A*SLO_MISS_ON_READS*# Reads+B*SLO_MISS_ON_WRITES*# Writes+C*CPU_CYCLES_ON_READ*reads+D*CPU_CYCLES_ON_WRITES*# Writes+E*MEMORY_REQUIREMENT*DataSetSize+F*(1/COMPRESSION_RATIO)*DataSetSize
wherein the parameters SLO_MISS_ON_READS comprises a value regarding how much the required SLO is missed when placing the data set in that storage area, SLO_MISS_ON_WRITES comprises a value regarding much the required SLO is missed when placing the data set in that storage area, CPU_CYCLES_ON_READS comprises a number of CPU cycles required for each read, CPU_CYCLES_ON_WRITES comprises a number of CPU cycles required for each write, MEMORY_REQUIREMENT comprises an amount of memory required to store the data set, COMPRESSION_RATIO comprises a compression ratio for the data set in the storage area, and wherein A-F comprise factors for adjusting a weight given to each parameter.
4. The method of claim 1 wherein each of the storage devices has an associated resource utilization level, the method further comprising adjusting a factor for a parameter when the resource utilization level exceeds a predetermined value.
5. The method of claim 4 wherein each resource utilization level has a predetermined number of utilization percentage steps and wherein when a resource utilization level exceeds a current step, then adjusting the factor for that parameter;
wherein the adjusting the factor for the parameter when the resource utilization level exceeds the predetermined value, in conjunction with the desired combination of compression and storage device type, further prevents overutilization of the resources.
6. The method of claim 1 wherein the storage device types are classified according to:
storage device bandwidth; and
writes per day.
7. A system for multi-dimensional determination of data placement in storage devices of a storage system, the system comprising:
a processor; and
memory storing computer program code that when executed on the processor causes the processor to operate a storage system, the storage system operable to perform the operations of:
ordering a plurality of data sets, wherein data sets identified as having a greater number of reads and writes than others of the data sets are placed higher in the ordering;
defining a plurality of storage areas based on storage device types of the storage devices, associated compression algorithms, and a plurality of parameters associated with different properties of the storage devices and the compression algorithms being used; and
storing a first data set of the plurality of data sets in a selected storage area of one of the storage devices based on a determination of which of the storage areas provides a desired combination of compression and storage device type for the first data set; and
storing, in another selected storage area of one of the storage devices, a next data set of the plurality of data sets based on the ordering, the storing the next data set is based on a determination of which of the storage areas provides a desired combination of compression and storage device type for the next data set;
wherein utilization of resources of the storage system is a function of the desired combination, and the desired combination is configured to prevent overutilization of the resources.
8. The system of claim 7 wherein ordering a plurality of data sets further comprises ordering the data sets according to a service level objective (SLO) associated with each of the plurality of data sets, a number of writes associated with each of the plurality of data sets, and a number of reads associated with each of the plurality of data sets.
9. The system of claim 7 wherein the selected storage area is determined according to a formula comprising:
A*SLO_MISS_ON_READS*# Reads+B*SLO_MISS_ON_WRITES*# Writes+C*CPU_CYCLES_ON_READ*reads+D*CPU_CYCLES_ON_WRITES*# Writes+E*MEMORY_REQUIREMENT*DataSetSize+F*(1/COMPRESSION_RATIO)*DataSetSize
wherein the parameters SLO_MISS_ON_READS comprises a value regarding how much the required SLO is missed when placing the data set in that storage area, SLO_MISS_ON_WRITES comprises a value regarding much the required SLO is missed when placing the data set in that storage area, CPU_CYCLES_ON_READS comprises a number of CPU cycles required for each read, CPU_CYCLES_ON_WRITES comprises a number of CPU cycles required for each write, MEMORY_REQUIREMENT comprises an amount of memory required to store the data set, COMPRESSION_RATIO comprises a compression ratio for the data set in the storage area, and wherein A-F comprise factors for adjusting a weight given to each parameter.
10. The system of claim 7 wherein each of the storage devices has an associated resource utilization level, the operations further comprising adjusting a factor for a parameter when the resource utilization level exceeds a predetermined value.
11. The system of claim 10 wherein each resource utilization level has a predetermined number of utilization percentage steps and wherein when a resource utilization level exceeds a current step, then adjusting the factor for that parameter;
wherein the adjusting the factor for the parameter when the resource utilization level exceeds the predetermined value, in conjunction with the desired combination of compression and storage device type, further prevents overutilization of the resources.
12. A computer program product for multi-dimensional determination of data placement in storage devices of a storage system, the computer program product including a non-transitory computer readable storage medium having computer program code encoded thereon that when executed on a processor of a computer causes the computer to operate a storage system, the computer program product comprising:
computer program code for ordering a plurality of data sets, wherein data sets identified as having a greater number of reads and writes than others of the data sets are placed higher in the ordering;
computer program code for defining a plurality of storage areas based on storage device types of the storage devices, associated compression algorithms, and a plurality of parameters associated with different properties of the storage devices and the compression algorithms being used;
computer program code for storing a first data set of the plurality of data sets in a selected storage area of one of the storage devices based on a determination of which of the storage areas provides a desired combination of compression and storage device type for the first data set; and
computer program code for storing, in another selected storage area of one of the storage devices, a next data set of the plurality of data sets based on the ordering, the storing the next data set is based on a determination of which of the storage areas provides a desired combination of compression and storage device type for the next data set;
wherein utilization of the resources of the storage system is a function of the desired combination, and the desired combination is configured to prevent overutilization of the resources.
13. The computer program product of claim 12 further comprising computer program code for ordering a plurality of data sets further comprises ordering the data sets according to a service level objective (SLO) associated with each of the plurality of data sets, a number of writes associated with each of the plurality of data sets, and a number of reads associated with each of the plurality of data sets.
14. The computer program product of claim 12 further comprising computer program code wherein the selected storage area is determined according to a formula comprising:
A*SLO_MISS_ON_READS*# Reads+B*SLO_MISS_ON_WRITES*# Writes+C*CPU_CYCLES_ON_READ*reads+D*CPU_CYCLES_ON_WRITES*# Writes+E*MEMORY_REQUIREMENT*DataSetSize+F*(1/COMPRESSION_RATIO)*DataSetSize
wherein the parameters SLO_MISS_ON_READS comprises a value regarding how much the required SLO is missed when placing the data set in that storage area, SLO_MISS_ON_WRITES comprises a value regarding much the required SLO is missed when placing the data set in that storage area, CPU_CYCLES_ON_READS comprises a number of CPU cycles required for each read, CPU_CYCLES_ON_WRITES comprises a number of CPU cycles required for each write, MEMORY_REQUIREMENT comprises an amount of memory required to store the data set, COMPRESSION_RATIO comprises a compression ratio for the data set in the storage area, and wherein A-F comprise factors for adjusting a weight given to each parameter.
15. The computer program product of claim 12 wherein each of the storage devices has an associated resource utilization level, the computer program product further comprising computer program code for adjusting a factor for a parameter when the resource utilization level exceeds a predetermined value.
16. The computer program product of claim 15 further comprising computer program code for adjusting a factor for a parameter when the resource utilization level exceeds a predetermined value, and wherein each resource utilization level has a predetermined number of utilization percentage steps and wherein when a resource utilization level exceeds a current step, then adjusting the factor for that parameter;
wherein the adjusting the factor for the parameter when the resource utilization level exceeds the predetermined value, in conjunction with the desired combination of compression and storage device type, further prevents overutilization of the resources.