Efficient packing of compressed data in storage system implementing data striping
An apparatus comprises at least one processing device comprising a processor coupled to a memory. The processing device is configured to select a stripe column size for stripes of a data storage system, to determine a first compress block size for a first one of the stripes based on compressibility of data to be stored, to select a first prime number for computing parity blocks for the first stripe and a first number of sub-stripes for splitting stripe columns of the first stripe, to generate metadata specifying the first compress block size, the first prime number and the first number of sub-stripes for the first stripe, and to store data compressed using the first compress block size in the first stripe. The first prime number and first number of sub-stripes for the first stripe is different than a second prime number and second number of sub-stripes for a second stripe.
1. An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the processing device being configured:
to select, for a storage system implementing data striping across a plurality of storage devices, a stripe column size for stripes for each of the plurality of storage devices;
to determine, for a first one of the stripes, a first compress block size based at least in part on compressibility of data to be stored in the first stripe;
to select, based at least in part on the first compress block size, a first prime number for computing parity blocks for the first stripe and a first number of sub-stripes for splitting stripe columns of the first stripe on each of the plurality of storage devices;
to generate, for the first stripe, metadata specifying the first compress block size, the first prime number and the first number of sub-stripes; and
to store, in the first stripe, data compressed using the first compress block size;
wherein at least one of the first compress block size, the first prime number and the first number of sub-stripes for the first stripe is different than at least one of a second compress block size, a second prime number and a second number of sub-stripes for a second one of the stripes of the storage system.
2. The apparatus of claim 1 wherein the storage system comprises a Redundant Array of Independent Disks (RAID) storage system.
3. The apparatus of claim 2 wherein a number of data disks in the RAID storage system comprises a prime number k, and wherein a column of the RAID storage system comprises k−1 blocks.
4. The apparatus of claim 2 wherein the plurality of storage devices comprise solid state drives (SSDs).
5. The apparatus of claim 2 wherein the RAID storage system implements RAID 6 with the number of data disks being k and the number of parity disks being n, where n is greater than one.
6. The apparatus of claim 1 wherein selecting the stripe column size comprises selecting the stripe column size as a multiple of a designated block size.
7. The apparatus of claim 6 wherein the first compress block size divides the designated block size with remainder.
8. The apparatus of claim 6 wherein the multiple of the designated block size comprises a (P−1) multiple of the designated block size where P is a prime number, and wherein the first prime number selected for the first stripe is different than P.
9. The apparatus of claim 1 wherein selecting the first prime number and the first number of sub-stripes comprises selecting a value of the first prime number to reduce a difference between: (i) the selected stripe column size; and (ii) the first compress block size multiplied by the first number of stripes multiplied by the first prime number.
10. The apparatus of claim 1 wherein the first prime number and the second prime number are the same, and wherein the first number of sub-stripes and the second number of sub-stripes are different.
11. The apparatus of claim 1 wherein when the first prime number is greater than a number of the plurality of storage devices in the storage system storing data blocks for the first stripe, parity blocks are computed utilizing a set of virtual storage devices that store pages with a designated predetermined value.
12. The apparatus of claim 11 wherein the designated predetermined value is zero.
13. The apparatus of claim 11 wherein a number of virtual storage devices in the set is equal to a difference between the first prime number and the number of storage devices in the storage system storing data blocks.
14. The apparatus of claim 1 wherein the first prime number is selected to reduce waste associated with storing compressed pages using the first compress block size in the selected stripe column size.
15. A method comprising:
selecting, for a storage system implementing data striping across a plurality of storage devices, a stripe column size for stripes for each of the plurality of storage devices;
determining, for a first one of the stripes, a first compress block size based at least in part on compressibility of data to be stored in the first stripe;
selecting, based at least in part on the first compress block size, a first prime number for computing parity blocks for the first stripe and a first number of sub-stripes for splitting stripe columns of the first stripe on each of the plurality of storage devices;
generating, for the first stripe, metadata specifying the first compress block size, the first prime number and the first number of sub-stripes; and
storing, in the first stripe, data compressed using the first compress block size;
wherein at least one of the first compress block size, the first prime number and the first number of sub-stripes for the first stripe is different than at least one of a second compress block size, a second prime number and a second number of sub-stripes for a second one of the stripes of the storage system; and
wherein the method is implemented by at least one processing device comprising a processor coupled to a memory.
16. The method of claim 15 wherein selecting the first prime number and the first number of sub-stripes comprises selecting a value of the first prime number to reduce a difference between: (i) the selected stripe column size; and (ii) the first compress block size multiplied by the first number of stripes multiplied by the first prime number.
17. The method of claim 15 wherein selecting the stripe column size comprises selecting the stripe column size as a multiple of a designated block size, and wherein the first compress block size divides the designated block size with remainder.
18. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes said at least one processing device:
to select, for a storage system implementing data striping across a plurality of storage devices, a stripe column size for stripes for each of the plurality of storage devices;
to determine, for a first one of the stripes, a first compress block size based at least in part on compressibility of data to be stored in the first stripe;
to select, based at least in part on the first compress block size, a first prime number for computing parity blocks for the first stripe and a first number of sub-stripes for splitting stripe columns of the first stripe on each of the plurality of storage devices;
to generate, for the first stripe, metadata specifying the first compress block size, the first prime number and the first number of sub-stripes; and
to store, in the first stripe, data compressed using the first compress block size;
wherein at least one of the first compress block size, the first prime number and the first number of sub-stripes for the first stripe is different than at least one of a second compress block size, a second prime number and a second number of sub-stripes for a second one of the stripes of the storage system.
19. The computer program product of claim 18 wherein selecting the first prime number and the first number of sub-stripes comprises selecting a value of the first prime number to reduce a difference between: (i) the selected stripe column size; and (ii) the first compress block size multiplied by the first number of stripes multiplied by the first prime number.
20. The computer program product of claim 18 wherein selecting the stripe column size comprises selecting the stripe column size as a multiple of a designated block size, and wherein the first compress block size divides the designated block size with remainder.