Managing inline data compression and deduplication in storage systems

1. A method for use in managing inline data compression and deduplication in storage systems, the method comprising:

identifying, based on entropy, a block of data from data stored in a cache of a storage system;

comparing entropy of the block of data with a first threshold value;

based on the comparison, either deduplicating the block of data or compressing the block of data without deduplication.

2. The method of claim 1 , wherein identifying the block of data comprises:

identifying between 4 KB and 128 KB of data to include in the block.

3. The method of claim 1 , wherein identifying the block of data comprises:

determining entropy of chunks of data stored in the cache;

including, in the block of data, chunks of data with entropy falling below a second threshold value.

4. The method of claim 3 , wherein a chunk includes 4 KB of data.

5. The method of claim 3 , wherein a chunk includes 8 KB of data.

6. The method of claim 3 , wherein comparing the entropy of the block of data with the first threshold value comprises:

determining the entropy of the block by averaging the entropy of chunks of data in the block.

7. The method of claim 1 , wherein identifying the block of data comprises:

identifying chunks of data stored in the cache within a predetermined window of time.

8. The method of claim 1 , wherein deduplicating the block of data comprises:

deduplicating the block of data in increments of 512 B.

9. The method of claim 1 , further comprising:

determining entropy of the deduplicated block of data;

comparing the entropy of the deduplicated block of data with a third threshold value;

based on the comparison, either compressing the deduplicated block of data or writing the deduplicated block of data to storage without compression.

10. The method of claim 9 , wherein determining the entropy of the deduplicated block of data comprises:

averaging the entropy of remaining data in the deduplicated block of data.

11. A system for use in managing inline data compression and deduplication in storage systems, the system comprising a processor configured to:

identify, based on entropy, a block of data from data stored in a cache of a storage system;

compare entropy of the block of data with a first threshold value;

based on the comparison, either deduplicate the block of data or compress the block of data without deduplication.

12. The system of claim 11 , wherein the processor is further configured to:

identify between 4 KB and 128 KB of data to include in the block.

13. The system of claim 11 , wherein the processor is further configured to:

determine entropy of chunks of data stored in the cache;

include, in the block of data, chunks of data with entropy falling below a second threshold value.

14. The system of claim 13 , wherein a chunk includes 4 KB of data.

15. The system of claim 13 , wherein a chunk includes 8 KB of data.

16. The system of claim 11 , wherein the processor is further configured to:

determine the entropy of the block by averaging the entropy of chunks of data in the block.

17. The system of claim 11 , wherein the processor is further configured to:

identify chunks of data stored in the cache within a predetermined window of time.

18. The system of claim 11 , wherein the processor is further configured to:

deduplicate the block of data in increments of 512 B.

19. The system of claim 11 , wherein the processor is further configured to:

determine entropy of the deduplicated block of data;

compare the entropy of the deduplicated block of data with a third threshold value;

based on the comparison, either compress the deduplicated block of data or write the deduplicated block of data to storage without compression.

20. The system of claim 19 , wherein the processor is further configured to:

average the entropy of remaining data in the deduplicated block of data.