Managing known data patterns for deduplication

1. A system, comprising:

a processor configured to:

determine that a reference count associated with a data pattern meets a promotion threshold value, wherein the reference count associated with the data pattern comprises a number of times that a data block associated with the data pattern is associated with one or more files;

in response to the determination that the reference count associated with the data pattern meets the promotion threshold value, promote the data pattern to a promoted data pattern, wherein the promoted data pattern is configured to be compared against data patterns of new data blocks to perform deduplication;

receive a new write request to write the data block to a file;

determine that the data pattern associated with the data block matches the promoted data pattern; and

perform the new write request by associating the file with a previously stored copy of the data block without updating the reference count associated with the data pattern; and

a memory coupled to the processor and configured to store the new write request itself.

2. The system of claim 1 , wherein the processor is further configured to:

receive a set of promoted data patterns, wherein:

data patterns included in deduplication data entries and associated reference counts are obtained from a plurality of storage systems; and

a subset of the data patterns are identified as the set of promoted data patterns based at least in part on those of the reference counts associated with the subset of the data patterns meeting the promotion threshold value.

3. The system of claim 1 , wherein the processor is further configured to:

receive an indication that a promotion event has occurred;

determine reference counts associated with data patterns included in deduplication data entries stored at the system;

identify a subset of the data patterns included in the deduplication data entries as promoted data patterns at least in part on those of the reference counts associated with the subset of the data patterns meeting the promotion threshold value; and

add the promoted data patterns into a known data pattern data structure.

4. The system of claim 3 , wherein the promotion threshold value is adjustable in response to at least one of: a current workload associated with the system and a remaining capacity at the known data pattern data structure.

5. The system of claim 1 , wherein the processor is further configured to:

determine a physical address associated with a location on persistent storage at which the previously stored copy of the data block associated with the promoted data pattern is stored;

store the physical address in an augmented block metadata record, wherein the augmented block metadata record comprises a data structure that stores the physical address associated with the location on the persistent storage at which the previously stored copy of the data block associated with the promoted data pattern is stored, wherein the augmented block metadata record does not store the reference count associated with the promoted data pattern; and

store, in a new entry in an auxiliary data structure, a corresponding relationship between a static representation associated with the promoted data pattern and a reference to the augmented block metadata record.

6. The system of claim 1 , wherein to perform the new write request by associating the file with the previously stored copy of the data block without updating the reference count comprises to store a static representation associated with the data block to a portion associated with the file that is specified by the new write request, wherein the static representation is usable to locate the previously stored copy of the data block.

7. The system of claim 6 , wherein the processor is further configured to:

receive a read request to the portion associated with the file;

read, from the portion associated with the file, the static representation associated with the data block;

use the static representation to perform a lookup in an auxiliary data structure;

obtain a reference to an augmented block metadata record based at least in part on a matching entry in the auxiliary data structure; and

obtain the data block from storage based at least in part on a physical address at which the previously stored copy of the data block was stored included in the augmented block metadata record.

8. The system of claim 6 , wherein the processor is further configured to:

receive a read request to the portion associated with the file;

read, from the portion associated with the file, the static representation associated with the data block;

use the static representation to perform a lookup in an auxiliary data structure; and

obtain the data block from storage based at least in part on a physical address at which the previously stored copy of the data block was stored from a matching entry in the auxiliary data structure.

9. A method, comprising:

determining that a reference count associated with a data pattern meets a promotion threshold value, wherein the reference count associated with the data pattern comprises a number of times that a data block associated with the data pattern is associated with one or more files;

in response to the determination that the reference count associated with the data pattern meets the promotion threshold value, promoting the data pattern to a promoted data pattern, wherein the promoted data pattern is configured to be compared against data patterns of new data blocks to perform deduplication;

receiving a new write request to write the data block to a file;

determining that the data pattern associated with the data block matches the promoted data pattern; and

performing the new write request by associating the file with a previously stored copy of the data block without updating the reference count associated with the data pattern.

10. The method of claim 9 , further comprising:

receiving an indication that a promotion event has occurred;

determining reference counts associated with data patterns included in deduplication data entries stored at a system;

identifying a subset of the data patterns included in the deduplication data entries as promoted data patterns at least in part on those of the reference counts associated with the subset of the data patterns meeting the promotion threshold value; and

adding the promoted data patterns into a known data pattern data structure.

11. The method of claim 9 , further comprising:

determining a physical address associated with a location on persistent storage at which the previously stored copy of the data block associated with the promoted data pattern is stored;

and

storing the physical address in an augmented block metadata record, wherein the augmented block metadata record comprises a data structure that stores the physical address associated with the location on the persistent storage at which the previously stored copy of the data block associated with the promoted data pattern is stored, wherein the augmented block metadata record does not store the reference count associated with the promoted data pattern.

12. The method of claim 9 , further comprising:

receiving a set of promoted data patterns, wherein:

data patterns included in deduplication data entries and associated reference counts are obtained from a plurality of storage systems; and

a subset of the data patterns are identified as the set of promoted data patterns based at least in part on those of the reference counts associated with the subset of the data patterns meeting the promotion threshold value.

13. The method of claim 10 , wherein the promotion threshold value is adjustable in response to at least one of: a current workload associated with the system and a remaining capacity at the known data pattern data structure.

14. The method of claim 9 , wherein performing the new write request by associating the file with the previously stored copy of the data block without updating the reference count comprises storing a static representation associated with the data block to a portion associated with the file that is specified by the new write request, wherein the static representation is usable to locate the previously stored copy of the data block.

15. The method of claim 14 , further comprising:

receiving a read request to the portion associated with the file;

reading, from the portion associated with the file, the static representation associated with the data block;

using the static representation to perform a lookup in an auxiliary data structure;

obtaining a reference to an augmented block metadata record based at least in part on a matching entry in the auxiliary data structure; and

obtaining the data block from storage based at least in part on a physical address at which the previously stored copy of the data block was stored included in the augmented block metadata record.

16. The method of claim 14 , further comprising:

receiving a read request to the portion associated with the file;

reading, from the portion associated with the file, the static representation associated with the data block;

using the static representation to perform a lookup in an auxiliary data structure; and

obtaining the data block from storage based at least in part on a physical address at which the previously stored copy of the data block was stored from a matching entry in the auxiliary data structure.