Restoring temporal locality in global and local deduplication storage systems
Techniques and mechanisms described herein facilitate the restoration temporal locality in global and local deduplication storage systems. According to various embodiments, when it is determined that cache memory in a storage system has reached a capacity threshold, each of a plurality of data dictionary entries stored in the cache memory may be associated with a respective merge identifier. Each data dictionary entry may correspond with a respective data chunk. Each data dictionary entry may indicate a storage location of the respective data chunk in the storage system. The respective merge identifier may indicate temporal locality information about the respective data chunk. The plurality of data dictionary entries may be stored to disk memory in the storage system. Each of the stored plurality of data dictionary entries may include the respective merge identifier.
1. A method comprising:
when it is determined that cache memory in a storage system has reached a capacity threshold, associating each of a plurality of data dictionary entries stored in the cache memory with a respective merge identifier, each data dictionary entry corresponding with a respective data chunk, each data dictionary entry indicating a storage location of the respective data chunk in the storage system, the respective merge identifier indicating temporal locality information about the respective data chunk; and
storing the plurality of data dictionary entries to disk memory in the storage system, each of the stored plurality of data dictionary entries including the respective merge identifier, the data dictionary entries stored to disk at the same time sharing the same merge identifier, the merge identifier indicating when the data dictionary entries were stored to disk.
2. The method recited in claim 1 , wherein the respective data chunks corresponding with the plurality of data dictionary entries are associated with a common data storage task, the data storage task corresponding with a request to store a set of interrelated data to the storage system.
3. The method recited in claim 1 , the method further comprising:
designating each of the plurality of data dictionary entries stored in the cache memory as stored to disk; and
overwriting at least a portion of the plurality of data dictionary entries stored in the cache memory with different data.
4. The method recited in claim 1 , wherein each data dictionary entry includes a data chunk identifier, the data chunk identifier uniquely identifying the respective data chunk in the storage system, the data chunk identifier being determined by applying a hash function to the respective data chunk corresponding with the data dictionary entry.
5. The method recited in claim 1 , wherein the storage system is configured to perform deduplication operations to reduce storage of duplicate data chunks on the storage system.
6. The method recited in claim 1 , wherein the storage system includes a plurality of storage nodes in communication via a network.
7. A method comprising:
receiving a request to retrieve a designated data dictionary entry in a storage system, the designated data dictionary entry corresponding with a designated data chunk, the designated data dictionary entry indicating a storage location of the designated data chunk in the storage system, the designated data dictionary entry being associated with a designated merge identifier, the designated merge identifier indicating temporal locality information about the designated data chunk;
identifying a plurality of data dictionary entries that are each associated with the designated merge identifier, each of the plurality of data dictionary entries corresponding with a respective data chunk, the plurality of data dictionary entries including the designated data dictionary entry; and
storing the plurality of data dictionary entries to cache memory associated with the storage system, the data dictionary entries stored to disk at the same time sharing the same merge identifier, the merge identifier indicating when the data dictionary entries were stored to disk.
8. The method recited in claim 7 , wherein the respective data chunks corresponding with the plurality of data dictionary entries are associated with a common data storage task, the data storage task corresponding with a request to store a set of interrelated data to the storage system.
9. The method recited in claim 7 , wherein identifying the plurality of data dictionary entries comprises transmitting request via a network to a remote storage node.
10. The method recited in claim 7 , wherein each of the plurality of data dictionary entries includes a respective data chunk identifier, the respective data chunk identifier uniquely identifying the respective data chunk in the storage system, the respective data chunk identifier being determined by applying a hash function to the respective data chunk corresponding with the data dictionary entry.
11. The method recited in claim 7 , wherein the storage system is configured to perform deduplication operations to reduce storage of duplicate data chunks on the storage system.
12. The method recited in claim 7 , wherein the storage system includes a plurality of storage nodes in communication via a network.
13. The one or more computer readable media having instructions stored thereon for performing a method, the method comprising:
when it is determined that cache memory in a storage system has reached a capacity threshold, associating each of a plurality of data dictionary entries stored in the cache memory with a respective merge identifier, each data dictionary entry corresponding with a respective data chunk, each data dictionary entry indicating a storage location of the respective data chunk in the storage system, the respective merge identifier indicating temporal locality information about the respective data chunk; and
storing the plurality of data dictionary entries to disk memory in the storage system, each of the stored plurality of data dictionary entries including the respective merge identifier, the data dictionary entries stored to disk at the same time sharing the same merge identifier, the merge identifier indicating when the data dictionary entries were stored to disk.
14. The one or more computer readable media recited in claim 13 , wherein the respective data chunks corresponding with the plurality of data dictionary entries are associated with a common data storage task, the data storage task corresponding with a request to store a set of interrelated data to the storage system.
15. The one or more computer readable media recited in claim 13 , the method further comprising:
designating each of the plurality of data dictionary entries stored in the cache memory as stored to disk; and
overwriting at least a portion of the plurality of data dictionary entries stored in the cache memory with different data.