Opportunistic asynchronous deduplication using an in-memory cache

1. A method for asynchronous deduplication of storage using an in-memory hash cache, said method comprising:

receiving, by a computing device, a plurality of input/output (I/O) commands, said plurality of commands including content subdivided into a plurality of blocks of data;

writing the plurality of blocks both to storage and to the in-memory hash cache prior to performing asynchronous deduplication; and

subsequently performing asynchronous deduplication of the storage using the in-memory hash cache while the plurality of blocks is stored in both the storage and the in-memory hash cache.

2. The method of claim 1 , wherein performing the asynchronous deduplication occurs upon availability of processing resources associated with the computing device.

3. The method of claim 1 , wherein performing the asynchronous deduplication occurs without performing read operations on the storage.

4. The method of claim 1 , wherein performing the asynchronous deduplication comprises:

calculating a hash of the blocks from the in-memory hash cache;

determining whether the calculated hash is available within a chunk store; and

updating a key-value table associated with the chunk store based on the determining.

5. The method of claim 4 , wherein updating the key-value table further comprises searching the key-value table for the calculated hash of the block.

6. The method of claim 5 , wherein updating the key-value table further comprises inserting the hash of the block as a key into the key-value table, and inserting a reference counter and an address of the block in the storage as the associated value of that key in the key-value table, upon the search failing to locate the hash in the key-value table.

7. The method of claim 4 , wherein updating the key-value table further comprises incrementing a reference counter of the hash of the block upon the search locating the hash in the key-value table.

8. The method of claim 4 , further comprising inserting a calculated hash of the blocks into the in-memory hash cache upon availability of the processing resources.

9. The method of claim 4 , wherein calculating the hash of the blocks is performed without reading the blocks from storage.

10. A non-transitory computer readable storage medium having stored thereon computer software executable by a processor, the computer software embodying a method for asynchronous deduplication of storage using an in-memory hash cache by:

receiving, by a computing device, a plurality of input/output (I/O) commands, said plurality of commands including content subdivided into a plurality of blocks of data;

writing the plurality of blocks both to storage and to the in-memory hash cache prior to performing opportunistic asynchronous deduplication; and

subsequently performing, while the plurality of blocks is stored in both the storage and the in-memory hash cache, opportunistic asynchronous deduplication of the storage using the in-memory hash cache upon availability of processing resources.

11. The non-transitory computer readable storage medium of claim 10 , wherein the computer software, upon execution, further evicts blocks from the in-memory hash cache after calculating a hash of those blocks.

12. The non-transitory computer readable storage medium of claim 10 , wherein the computer software, upon execution, performs the opportunistic deduplication without performing read operations on the storage.

13. The non-transitory computer readable storage medium of claim 10 , wherein the computer software, upon execution, evicts blocks from the in-memory hash cache when the in-memory hash cache is full.

14. A system for asynchronous deduplication of storage using an in-memory hash cache, said system comprising:

an in-memory hash cache of a computing device;

an input/output (I/O) stack of the computing device, the I/O stack programmed to:

receive a plurality of I/O commands, said plurality of commands including content subdivided into a plurality of blocks of data; and

write the plurality of blocks both to storage and to the in-memory hash cache prior to performing opportunistic asynchronous deduplication, the storage being remote from the computing device; and

a deduplication thread executing on the computing device to perform, while the plurality of blocks is stored in both the storage and the in-memory hash cache, opportunistic asynchronous deduplication of the storage using the in-memory hash cache upon availability of processing resources associated with the computing device.

15. The system of claim 14 , wherein a latency for accessing the in-memory hash cache is less than a latency for accessing the storage.

16. The system of claim 14 , wherein the in-memory hash cache stores the plurality of blocks and hashes of the plurality of blocks.

17. The system of claim 14 , further comprising a memory area storing a key-value table, the key-value table including hashes of the plurality of blocks, a reference counter, and an address, in storage, of the plurality of blocks.

18. The system of claim 14 , wherein the in-memory hash cache is flash memory and the storage is cloud storage.

19. The system of claim 14 , wherein the deduplication thread executes to perform opportunistic asynchronous deduplication by accessing the plurality of blocks from the in-memory hash cache instead of from the storage.

20. The system of claim 14 , wherein the deduplication thread executes separately from the I/O stack.