Optimized block storage for change block tracking systems
A system for backup comprises an input interface configured to receive a request to back up a file. The system for backup additionally comprises a processor configured to determine whether the file comprises an indicator to back up without deduplication, and in the event the file comprises an indicator to back up without deduplication, process the file for deduplication storage without deduplication.
1. A system for backup, comprising:
an input interface configured to receive a request to back up a file; and
a processor configured to:
in response to receiving the request to back up the file, determine whether use deduplication in connection with backing up the file, wherein to determine whether to use deduplication comprises to determine whether the file comprises an indicator that indicates that the file is to be backed up without deduplication, wherein the indicator that indicates that the file is to be backed up without deduplication is associated with the file based on a determination of whether the file is to be backed up without deduplication, and wherein the indicator is set with respect to the file to indicate that the file is to be backed up without deduplication based at least in part on a determination of whether corresponding blocks comprise only new data;
if the file comprises the indicator to back up without deduplication,
process the file for deduplication storage without deduplication, including obtaining a unique identifier associated with the file and associating the unique identifier with the file, wherein association of the unique identifier with the file indicates that the file is stored without deduplication; and
if the file does not comprise the indicator to back up without deduplication,
process the file for deduplication storage with deduplication, including obtaining a finger print associated with the file by performing a hash function.
2. The system of claim 1 , wherein the file comprises a data file or a data block.
3. The system of claim 1 , wherein processing the file for deduplication storage without deduplication comprises determining whether the file comprises a null file.
4. The system of claim 3 , wherein processing the file for deduplication storage without deduplication comprises determining a global unique identifier in the event that the file comprises the null file.
5. The system of claim 4 , wherein processing the file for deduplication storage without deduplication comprises storing the file and the global unique identifier in the deduplicating storage.
6. The system of claim 1 , wherein processing the file for deduplication storage without deduplication comprises segmenting the file into one or more segments in the event that the file does not comprise a null file.
7. The system of claim 6 , wherein obtaining the unique identifier associated with the file comprises determining one or more global unique identifiers for each of the one or more segments.
8. The system of claim 7 , wherein processing the file for deduplication storage without deduplication comprises storing the one or more segments and the one or more global unique identifiers associated with each of the one or more segments.
9. The system of claim 8 , wherein processing the file for deduplication storage without deduplication comprises storing a metadata mapping, wherein the metadata mapping maps the one or more segments to the file.
10. The system of claim 1 , wherein processing the file for deduplication storage without deduplication comprises using one or more global unique identifiers associated with the file.
11. The system of claim 10 , wherein one of the one or more globally unique identifiers is selected from a set of identifiers.
12. The system of claim 10 , wherein one of the one or more globally unique identifiers is selected a range of identifiers.
13. The system of claim 10 , wherein one of the one or more globally unique identifiers is selected randomly from within a set of identifiers.
14. The system of claim 10 , wherein one of the one or more globally unique identifiers is incremented from a previously selected identifier.
15. The system of claim 10 , wherein one of the one or more globally unique identifiers indicates that the file is stored with deduplication.
16. The system of claim 10 , wherein one of the one or more globally unique identifiers indicates that the file is stored without deduplication.
17. The system of claim 10 , wherein one of the one or more globally unique identifiers indicates that the file is a null file.
18. The system of claim 1 , wherein the file comprises the indicator to backup without deduplication in response to a determination that the file corresponds to one or more blocks with new data.
19. The system of claim 1 , wherein the unique identifier associated with the file is used to determine whether at least part of the file has been previously stored.
20. The system of claim 1 , wherein to determine whether the file comprises the indicator to backup without deduplication does not require computing a finger print of the file.
21. The system of claim 1 , wherein the indicator does not correspond to a finger print of the file.
22. The system of claim 1 , wherein the unique identifier is globally unique and the unique identifier is obtained in a manner that is different from a manner in which the finger print for association with a file to be deduplicated is obtained.
23. The method of claim 1 , wherein in response to determining that the file comprises the indicator, the file is processed for deduplication storage without computing a finger print associated with the file.
24. The method of claim 1 , wherein a file comprising only new data further comprises the indicator to backup without deduplication.
25. A method for backup, comprising:
receiving a request to back up a file;
in response to receiving the request to back up the file, determining whether use deduplication in connection with backing up the file, wherein to determine whether to use deduplication comprises determining whether the file comprises an indicator that indicates that the file is to be backed up without deduplication, wherein the indicator that indicates that the file is to be backed up without deduplication is associated with the file based on a determination of whether the file is to be backed up without deduplication, and wherein the indicator is set with respect to the file to indicate that the file is to be backed up without deduplication based at least in part on a determination of whether corresponding blocks comprise only new data;
if the file comprises the indicator to back up without deduplication, processing the file for deduplication storage without deduplication, including obtaining a unique identifier associated with the file and associating the unique identifier with the file, wherein association of the unique identifier with the file indicates that the file is stored without deduplication; and
if the file does not comprise the indicator to back up without deduplication, processing the file for deduplication storage with deduplication, including obtaining a finger print associated with the file by performing a hash function.
26. The method of claim 25 , further comprising:
in response to receiving a request to recover the file, using the unique identifier associated with the file to reconstruct the file.
27. The method of claim 25 , further comprising:
determining whether to associate the indicator with the file, the indicator being indicative of whether the file is to be backed up without deduplication; and
in response to determining that the indicator that indicates that the file is to backed up without deduplication, associating the indicator with the file.
28. The method of claim 27 , wherein the determining whether to associate the indicator with the file is based at least in part on a determination of an extent of new data comprised in the file.
29. A computer program product for backup, the computer program product being embodied in a non-transitory tangible computer readable storage medium and comprising computer instructions for:
receiving a request to back up a file;
in response to receiving the request to back up the file, determining whether use deduplication in connection with backing up the file, wherein to determine whether to use deduplication comprises determining whether the file comprises an indicator that indicates that the file is to backed up without deduplication, wherein the indicator that indicates that the file is to be backed up without deduplication is associated with the file based on a determination of whether the file is to be backed up without deduplication, and wherein the indicator is set with respect to the file to indicate that the file is to be backed up without deduplication based at least in part on a determination of whether corresponding blocks comprise only new data;
if the file comprises the indicator to back up without deduplication, processing the file for deduplication storage without deduplication, including obtaining a unique identifier associated with the file and associating the unique identifier with the file, wherein association of the unique identifier with the file indicates that the file is stored without deduplication; and
if the file does not comprise the indicator to back up without deduplication, processing the file for deduplication storage with deduplication, including obtaining a finger print associated with the file by performing a hash function.