Adaptive source side deduplication
A backup of one or more objects is determined to be performed. Based on one or more conditions, a corresponding deduplication option among a plurality of deduplication options to utilize when backing up the one or more objects is selected. The one or more conditions at least include a condition based on a detected data change pattern. The plurality of deduplication options include a deduplication option associated with utilizing at least in part a plurality of variable-length data chunks for one or more mismatched ranges and/or one or more missing ranges associated with one of the one or more objects associated with the source system. A request to perform the backup of the one or more objects according to the corresponding selected deduplication option is provided to the source system. Backup data associated with the one or more objects is received and stored.
1 . A method, comprising:
determining a backup of one or more objects associated with a source system is to be performed;
based on one or more conditions for the one or more objects, selecting a corresponding deduplication option among a plurality of deduplication options to utilize when the backup of the one or more objects is performed, wherein the one or more conditions include a condition based on a data change pattern indicating at least one of one or more mismatch ranges for the one or more objects or one or more missing ranges for the one or more objects, and wherein the plurality of deduplication options include a deduplication option associated with utilizing a plurality of variable-length data chunks for one or more mismatched ranges and/or one or more missing ranges associated with the one or more objects;
providing to the source system a request to perform the backup of the one or more objects according to the corresponding selected deduplication option;
receiving, from the source system, backup data associated with the one or more objects, wherein the backup data comprises, according to the corresponding selected deduplication option, content data associated with the one or more objects or a plurality of data chunks associated with the one or more objects; and
storing the backup data associated with the one or more objects.
2 . The method of claim 1 , wherein the one or more conditions for the one or more objects further include resource availability, data change rate, distribution of data change across objects, and/or object size.
3 . The method of claim 1 , wherein selecting the corresponding deduplication option includes determining a historical size for an object of the one or more objects and comparing the historical size for the object to a threshold size.
4 . The method of claim 3 , wherein selecting the corresponding deduplication option includes selecting a first deduplication option that does not include source side deduplication being performed for the object based on determining that the historical size for the object is not greater than the threshold size.
5 . The method of claim 4 , wherein based on a request to perform a backup of the object according to the first deduplication option, the source system determines a current size of the object.
6 . The method of claim 5 , wherein the source system provides, to a storage system, data associated with the object based on a determination that the current size of the object is not greater than the threshold size.
7 . The method of claim 5 , wherein the source system performs the source side deduplication for the object based on a determination that the current size of the object is greater than the threshold size.
8 . The method of claim 3 , wherein selecting the corresponding deduplication option includes determining an object identifier for an object of the one or more objects based on determining that the historical size for the object is greater than the threshold size.
9 . The method of claim 8 , wherein selecting the corresponding deduplication option comprises:
determining a percentage of historical mismatched ranges associated with different backed up versions of an object of the one or more objects; and
comparing the percentage of historical mismatched ranges associated with the different backed up versions of the object to a threshold percentage.
10 . The method of claim 9 , wherein selecting the corresponding deduplication option includes selecting, for an object of the one or more objects, a second deduplication option based on determining that the percentage of historical mismatched ranges associated with different backed up versions of the object is greater than the threshold percentage.
11 . The method of claim 10 , wherein based on a request to perform a backup of the object according to the second deduplication option, the source system performs source side deduplication for the object utilizing variable-length data chunks.
12 . The method of claim 9 , wherein selecting the corresponding deduplication option includes:
selecting, for an object of the one or more objects, a third deduplication option that includes performing source side deduplication for one or more mismatched ranges and/or one or more missing ranges based on determining that the percentage of historical mismatched ranges associated with the different backed up versions of the object is not greater than the threshold percentage; and
providing range information associated with the object in a request to perform a backup of the object according to the third deduplication option.
13 . The method of claim 12 , wherein based on the request to perform the backup of the object according to the third deduplication option, the source system determines one or more mismatched ranges and/or one or more missing ranges associated with the object.
14 . The method of claim 13 , wherein the source system determines whether there are sufficient CPU resources to perform source side deduplication for the one or more determined mismatched ranges and/or the one or more missing ranges associated with the object.
15 . The method of claim 14 , wherein the source system performs the source side deduplication for the one or more determined mismatched ranges and/or the one or more missing ranges associated with the object based on determining that the source system has sufficient CPU resources to perform source side deduplication for the object.
16 . The method of claim 14 , wherein the source system provides content data of the one or more determined mismatched ranges and/or the one or more missing ranges associated with the object based on determining that the source system does not have sufficient CPU resources to perform source side deduplication for the object.
17 . The method of claim 12 , wherein the range information associated with the object includes one or more range chunk identifiers, a corresponding offset for each of the one or more range chunk identifiers, and a corresponding length for one or more data chunks associated with a range chunk identifier for each of the one or more range chunk identifiers.
18 . The method of claim 1 , wherein storing the backup data associated with the one or more objects includes performing deduplication on the backup data associated with the one or more objects.
19 . Non-transitory computer-readable media comprising computer instructions that, when executed by one or more processors, cause the one or more processors to:
determine a backup of one or more objects associated with a source system is to be performed;
based on one or more conditions for the one or more objects, select a corresponding deduplication option among a plurality of deduplication options to utilize when the backup of the one or more objects is performed, wherein the one or more conditions include a condition based on a data change pattern indicating at least one of one or more mismatch ranges or one or more missing ranges for the one or more objects, and wherein the plurality of deduplication options include a deduplication option associated with utilizing a plurality of variable-length data chunks for one or more mismatched ranges and/or one or more missing ranges associated with the one or more objects associated with the source system;
provide to the source system a request to perform the backup of the one or more objects according to the corresponding selected deduplication option;
receive, from the source system, backup data associated with the one or more objects, wherein the backup data comprises, according to the corresponding selected deduplication option, content data associated with the one or more objects or a plurality of data chunks associated with the one or more objects; and
store the backup data associated with the one or more objects.
20 . A system, comprising:
memory storing instructions;
a processor configured to execute the instructions to:
determine a backup of one or more objects associated with a source system is to be performed;
based on one or more conditions for the one or more objects, select a corresponding deduplication option among a plurality of deduplication options to utilize when the backup of the one or more objects is performed, wherein the one or more conditions include a condition based on a data change pattern indicating at least one of one or more mismatch ranges or one or more missing ranges for the one or more objects, and wherein the plurality of deduplication options include a deduplication option associated with utilizing a plurality of variable-length data chunks for one or more mismatched ranges and/or one or more missing ranges associated with the one or more objects;
provide to the source system a request to perform the backup of the one or more objects according to the corresponding selected deduplication option;
receive, from the source system, backup data associated with the one or more objects, wherein the backup data comprises, according to the corresponding selected deduplication option, content data associated with the one or more objects or a plurality of data chunks associated with the one or more objects; and
store the backup data associated with the one or more objects.