Cache data placement for compression in data storage systems
A technique for managing data storage in a data storage system is disclosed. Data blocks are written to a data storage system cache, pluralities of the data blocks being organized into cache macro blocks, the IO cache macro blocks having a fixed size. Access requests for the data blocks are processed, wherein processing includes generating block access statics. Using access statics, data blocks stored in the cache macroblocks having block a access times that overlap are identified. Data blocks identified as having overlapping access times are rearranged into one or more overlap cache macroblocks. Data storage system cache memory is arranged into multiple input/output (IO) cache macroblocks, where a first set of IO cache macroblocks are configured as compressed IO cache macro blocks, each compressed IO cache macro block storing a plurality of variable sized compressed IO data blocks, and a second set of IO cache macroblocks are configured as non-compressed IO cache macroblocks, each non-compressed IO cache macroblock storing a plurality of fixed sized non-compressed IO data blocks. A write request is receive at the data storage system. If the IO data associated with the write request is determined to be compressible, the IO data is compressed in-line and written to an IO data block in a compressed IO cache macroblock, otherwise non-compressed IO data is written to an IO data block in a non-compressed IO cache macroblock.
1. A method for managing data storage in a data storage system, the method comprising:
a. Writing a plurality of data blocks to a data storage system cache, wherein the data storage system cache has a first fixed size cache macroblock and a second fixed size cache macroblock;
b. Processing a first access request to read a first data block of the first fixed size cache macroblock;
c. Recording a first temporal access statistic associated with the first access request to read the first data block of the first fixed size cache macroblock;
d. Processing a second access request to read a second data block of the second fixed size cache macroblock;
e. Recording a second temporal access statistic associated with the second access request to read the second data block of the second fixed size cache macroblock;
f. Comparing the first temporal access statistic to the second temporal access statistic to determine if they overlap in time;
g. Identifying, based on said comparing, that the first data block of the first fixed size cache macroblock and the second data block of the second fixed size cache macroblock have overlapping access times; and
h. Rearranging data blocks identified as having overlapping access times into an overlap cache macroblock located in the data storage system cache, wherein said rearranging includes storing the first data block of the first fixed size cache macroblock and the second data block of the second fixed size cache macroblock identified as having overlapping access times in the overlap cache macroblock; and
i. Writing the overlap cache macroblock to a primary persistent storage and deleting the first temporal access statistic and the second temporal access statistic.
2. The method of claim 1 , further comprising writing the overlap cache macroblock to a back-end hard disk drive.
3. The method of claim 1 , further comprising:
a. determining that at least a portion of a persistent storage comprises a flash memory device; and
b. preventing copying a data block to the portion of the persistent storage comprising the flash memory device.
4. The method of claim 1 , further comprising deleting the overlap cache macroblock if it is not written to the primary persistent storage after a threshold time limit.
5. The method of claim 1 , further comprising creating a cache macroblock data structure, wherein the cache macroblock data structure includes metadata describing a plurality of access characteristics of the data blocks organized therein.
6. The method of claim 1 wherein the first fixed size cache macroblock is configured to store compressed data blocks and the second fixed size cache macroblock is configured to store uncompressed data blocks.
7. The method of claim 1 wherein the first fixed size cache macroblock is equal in size to the second fixed size cache macroblock.
8. A system for managing data storage in a data storage system, the system comprising a special purpose storage processor and memory configured to:
a. Write a plurality of data blocks to a data storage system cache, wherein the data storage system cache has a first fixed size cache macroblock and a second fixed size cache macroblock;
b. Process a first access request to read a first data block of the first fixed size cache macroblock;
c. Record a first temporal access statistic associated with the first access request to read the first data block of the first fixed size cache macroblock;
d. Process a second access request to read a second data block of the second fixed size cache macroblock;
e. Record a second temporal access statistic associated with the second access request to read the second data block of the second fixed size cache macroblock;
f. Compare the first temporal access statistic to the second temporal access statistic to determine if they overlap in time;
g. Identify, based on said comparing, that the first data block of the first fixed size cache macroblock and the second data block of the second fixed size cache macroblock have overlapping access times; and
h. Rearrange data blocks identified as having overlapping access times into an overlap cache macroblock located in the data storage system cache, wherein said rearranging includes storing the first data block of the first fixed size cache macroblock and the second data block of the second fixed size cache macroblock identified as having overlapping access times in the overlap cache macroblock; and
i. Write the overlap cache macroblock to a primary persistent storage and delete the first temporal access statistic and the second temporal access statistic.
9. The system of claim 8 wherein the special purpose storage processor and memory are further configured to write the overlap cache macroblock to a back-end hard disk drive.
10. The system of claim 8 wherein the special purpose storage processor and memory are further configured to determine whether at least a portion of a persistent storage comprises a flash memory device and prevent copying a data block to the portion of the persistent storage comprising the flash memory device.
11. The system of claim 8 wherein the special purpose storage processor and memory are further configured to delete the overlap cache macroblock if it is not written to the primary persistent storage after a threshold time limit.
12. The system of claim 8 wherein the special purpose storage processor and memory are further configured to create a cache macroblock data structure, wherein the cache macroblock data structure includes metadata describing a plurality of access characteristics of the data blocks organized therein.
13. The system of claim 8 wherein the first fixed size cache macroblock is configured to store compressed data blocks and the second fixed size cache macroblock is configured to store uncompressed data blocks.
14. The system of claim 8 wherein the first fixed size cache macroblock is equal in size to the second fixed size cache macroblock.
15. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method for managing data storage in a data storage system, the method comprising:
a. Writing a plurality of data blocks to a data storage system cache, wherein the data storage system cache has a first fixed size cache macroblock and a second fixed size cache macroblock;
b. Processing a first access request to read a first data block of the first fixed size cache macroblock;
c. Recording a first temporal access statistic associated with the first access request to read the first data block of the first fixed size cache macroblock;
d. Processing a second access request to read a second data block of the second fixed size cache macroblock;
e. Recording a second temporal access statistic associated with the second access request to read the second data block of the second fixed size cache macroblock;
f. Comparing the first temporal access statistic to the second temporal access statistic to determine if they overlap in time;
g. Identifying, based on said comparing, that the first data block of the first fixed size cache macroblock and the second data block of the second fixed size cache macroblock have overlapping access times; and
h. Rearranging data blocks identified as having overlapping access times into an overlap cache macroblock located in the data storage system cache, wherein said rearranging includes storing the first data block of the first fixed size cache macroblock and the second data block of the second fixed size cache macroblock identified as having overlapping access times in the overlap cache macroblock; and
i. Writing the overlap cache macroblock to a primary persistent storage and deleting the first temporal access statistic and the second temporal access statistic.