Efficient compressed track size classification to reduce disk fragmentation and increase probability of in-place compressed writes
In a data storage system in which a full-size allocation unit is used for storage of uncompressed data, an optimal reduced size allocation unit is selected for storage of compressed data. Changes in the compressed size of at least one full-size allocation unit of representative data are monitored over time. The representative data may be selected based on write frequency, relocation frequency, or both. Compression size values are counted and weighted to calculate the optimal reduced allocation unit size. The optimal reduced size allocation unit is used for storage of compressed data. A full-size allocation unit of data that cannot be accommodated by a reduced size allocation unit when compressed is stored uncompressed.
1. An apparatus comprising:
a data storage system comprising:
a plurality of compute nodes interconnected with a plurality of drives that are accessible by the compute nodes only in full-size allocation units and reduced size allocation units that are characterized by smaller size than the full-size allocation units, wherein the compute nodes are configured to use the full-size allocation units for uncompressed data and use the reduced-size allocation units for compressed data;
a storage object on which data is logically stored, the storage object data being physically stored by the drives; and
a compressed data manager configured to:
select at least one full-size allocation unit of representative data of the storage object;
monitor changes in compressibility of the representative data over time;
select a size for the reduced size allocation units for the storage object based on the changes in compressibility of the representative data over time;
cause the compute nodes to use the full-size allocation units for storage of uncompressed data of the storage object on the drives; and
cause the compute nodes to use the reduced size allocation units of the selected size for storage of only compressed data of the storage object on the drives.
2. The apparatus of claim 1 wherein the storage system uses only one size of reduced size allocation unit for storage of compressed data.
3. The apparatus of claim 1 wherein the compressed data manager weights a plurality of buckets corresponding to respective data sizes, and compares the data sizes of the weighted buckets to the monitored compressibility of the representative data to select the size for the reduced allocation units for the storage object.
4. The apparatus of claim 3 wherein each of the buckets has a Bucket Size that corresponds to a respective amount of compressibility of the representative data and an associated Bucket Size Counter Value that indicates how many times monitored changes in the compressibility of the representative data corresponded to the respective Bucket Size, the compressed data manager configured to weight the buckets using weights W=Bucket Size Counter Value*(Bucket Size/100) P , where P is a hyper parameter to select the size for the reduced size allocation units for the storage object.
5. The apparatus of claim 1 wherein the compressed data manager selects the at least one full-size allocation unit of representative data based on frequency of write operations.
6. The apparatus of claim 1 wherein the compressed data manager selects the at least one full-size allocation unit of representative data based on frequency of relocation operations.
7. The apparatus of claim 1 wherein the compressed data manager selects the reduced allocation unit size based on changes in average compressibility of full-size allocation units of data.
8. A method comprising:
accessing non-volatile drives only in full-size allocation units and reduced size allocation units that are characterized by smaller size than the full-size allocation units;
selecting at least one full-size allocation unit of representative data;
monitoring changes in compressibility of the representative data over time;
selecting a size for the reduced size allocation units based on the changes in compressibility of the representative data over time;
using full-size allocation units for storage of uncompressed data on the non-volatile drives; and
using reduced size allocation units for storage of only compressed data on the non-volatile drives.
9. The method of claim 8 comprising using only one size of reduced size allocation unit for storage of compressed data.
10. The method of claim 8 wherein selecting the size for the reduced size allocation unit based on the changes in the compressibility of the representative data over time comprises weighting a plurality of buckets corresponding to respective data sizes, and comparing the data sizes of the weighted buckets to the monitored compressibility of the representative data.
11. The method of claim 10 wherein each of the buckets has a Bucket Size that corresponds to a respective amount of compressibility of the representative data and an associated Bucket Size Counter Value that indicates how many times monitored changes in the compressibility of the representative data corresponded to the respective Bucket Size, and further comprising weighting the values of the monitored compressed size by calculating a weight W=Bucket Size Counter Value*(Bucket Size/100) P , where P is a hyper parameter.
12. The method of claim 8 wherein selecting the at least one full-size allocation unit of representative data comprises selecting based on frequency of write operations.
13. The method of claim 8 wherein selecting the at least one full-size allocation unit of representative data comprises selecting based on frequency of relocation operations.
14. The method of claim 8 comprising selecting the reduced size allocation unit size based on changes in average compressibility of full-size allocation units of data.
15. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for using a computer system to implement multiple sizes of allocation units for storage of data, the method comprising:
accessing non-volatile drives only in full-size allocation units and reduced size allocation units that are characterized by smaller size than the full-size allocation units;
selecting at least one full-size allocation unit of representative data;
monitoring changes in compressibility of the representative data over time;
selecting a size for the reduced size allocation units based on the changes in compressibility of the representative data over time;
using full-size allocation units for storage of uncompressed data on the non-volatile drives;
and using reduced size allocation units for storage of only compressed data on the non-volatile drives.
16. The non-transitory computer-readable storage medium of claim 15 comprising using only one size of reduced size allocation unit for storage of compressed data.
17. The non-transitory computer-readable storage medium of claim 15 wherein selecting the size for the reduced size allocation unit based on the changes in the compressibility of the representative data over time comprises weighting a plurality of buckets corresponding to respective data sizes, and comparing the data sizes of the weighted buckets to the monitored compressibility of representative data.
18. The non-transitory computer-readable storage medium of claim 15 wherein selecting the at least one full-size allocation unit of representative data comprises selecting based on frequency of write operations.
19. The non-transitory computer-readable storage medium of claim 15 wherein selecting the at least one full-size allocation unit of representative data comprises selecting based on frequency of relocation operations.
20. The non-transitory computer-readable storage medium of claim 15 comprising selecting the reduced size allocation unit size based on changes in average compressibility of full-size allocation units of data.