Deduplication rehydration
A method for deduplication rehydration is described. In one embodiment, a request to restore a backup image is received. The backup image is stored in a deduplication system. The backup image includes a plurality of data segments. The method includes determining locality information for at least one of the plurality of data segments. The locality information includes information regarding a location of the at least one data segment in relation to each other data segment of the plurality of data segments in the backup image. The method includes obtaining an identifier of each data container storing the plurality of data segments of the backup image, determining a degree to which the plurality of data segments of the backup image are processed by prefetching, and prefetching one or more of the plurality of target data segments from a data container based at least in part on a predetermined effectiveness threshold.
1. A method for deduplication rehydration, comprising:
receiving a request to restore a backup image, the backup image being stored in a deduplication system, wherein the backup image comprises a plurality of data segments;
determining locality information for at least one of the plurality of data segments, wherein the locality information comprises information regarding a location of the at least one data segment in relation to each other data segment of the plurality of data segments in the backup image;
obtaining an identifier of each data container storing the plurality of data segments of the backup image; and
determining a degree to which the plurality of data segments of the backup image are processed by prefetching.
2. The method of claim 1 , further comprising:
calculating a prefetch ratio based on a complementary percentage of a value of a predetermined self-read ratio, the prefetch ratio expressing the degree to which the plurality of data segments of the backup image are processed by prefetching.
3. The method of claim 1 , further comprising:
calculating a data segment sum, wherein calculating the data segment sum comprises summing a number of data segments in the backup image with a number of data segments currently being prefetched by all prefetch threads.
4. The method of claim 3 , further comprising:
calculating a data container sum, wherein calculating the data container sum comprises summing a number of unique data container files containing the plurality of data segments in the backup image with a number of unique data container files containing the data segments currently being prefetched by all prefetch threads.
5. The method of claim 4 , further comprising:
calculating a merged locality, wherein calculating the merged locality comprises determining a ratio of the data segment sum to the data container sum.
6. The method of claim 5 , further comprising:
if merged locality satisfies a predetermined high-locality threshold, then setting a self-read ratio of the request to restore a backup image to 100 percent, the self-read ratio expressing the degree to which the plurality of data segments of the backup image are processed without prefetching.
7. The method of claim 5 , further comprising:
if merged locality satisfies a predetermined low-locality threshold and fails to satisfy a predetermined high-locality threshold, then calculating a magnification factor based on a ratio of merged locality to the low-locality threshold; and
calculating a self-read ratio based on a product of the magnification factor and a predetermined self-read-ratio baseline value.
8. The method of claim 5 , further comprising:
if merged locality fails to satisfy both a predetermined low-locality threshold and a predetermined high-locality threshold, then calculating a magnification factor based on a ratio of the low-locality threshold to merged locality; and
calculating a self-read ratio based a complementary percentage of a product of the magnification factor and a complementary percentage of a predetermined self-read ratio-baseline value.
9. The method of claim 1 , further comprising:
determining a value for at least one of a low-locality threshold, a high-locality threshold, and a self-read-ratio baseline based at least in part on analyzing system parameters and accessing previous test data.
10. A method for deduplication rehydration, comprising:
receiving a request to restore a backup image, the backup image being stored in a deduplication system, wherein the backup image comprises a plurality of target data segments;
determining locality information for at least one of the plurality of target data segments, wherein the locality information comprises information regarding a location of the at least one target data segment in relation to each other target data segment of the plurality of target data segments in the backup image;
obtaining an identifier of each data container storing the plurality of target data segments of the backup image; and
prefetching one or more of the plurality of target data segments from a data container based at least in part on a predetermined effectiveness threshold.
11. The method of claim 10 , further comprising:
determining a maximum read length, wherein the maximum read length is based on an allocated memory size for a temporary buffer.
12. The method of claim 11 , further comprising:
selecting a data container, the data container containing one or more target data segments from the backup image.
13. The method of claim 12 , further comprising:
iteratively determining length values for a consecutive sequence of one or more data segments in the data container, comprising:
a first iteration comprising determining a length of a first target data segment in the data container;
adding at least one sequential data segment to a current length of the one or more data segments for each subsequent iteration;
determining a length of target data segments in each subsequent iteration; and
stopping the iterative determination of length values when a current length value exceeds the maximum read length or the current length of the one or more data segments includes all target data segments in the data container.
14. The method of claim 13 , further comprising:
saving the determined length values in an array of length values for each length value that does not exceed a determined length value of a final iteration.
15. The method of claim 14 , further comprising:
analyzing the one or more data segments within each consecutive sequence of one or more data segments from the final iteration length value to the shortest length value, wherein the analyzing the one or more data segments comprises determining a number of target data segments included within each consecutive sequence of one or more data segments; and
determining an effectiveness ratio for each consecutive sequence of one or more data segments, the effectiveness ratio comprising a ratio of a length of the one or more target data segments within a first consecutive sequence of one or more data segments to a length value of the first consecutive sequence of one or more data segments.
16. The method of claim 15 , further comprising:
wherein, if the effectiveness ratio of an analyzed length satisfies an effectiveness threshold and the effectiveness ratio of the analyzed length is the largest of the effectiveness ratios in the array of length values, then selecting the analyzed length; and
reading all of the data segments from the analyzed length into the temporary buffer.
17. The method of claim 16 , further comprising:
reading the target data segments in the temporary buffer into a shared buffer, wherein the shared buffer comprises each of the data segments of the backup image.
18. A computing device configured for improving deduplication rehydration, comprising:
a processor;
memory in electronic communication with the processor, wherein the memory stores computer executable instructions that when executed by the processor cause the processor to perform the steps of:
receiving a request to restore a backup image, the backup image being stored in a deduplication system, wherein the backup image comprises a plurality of data segments;
determining locality information for at least one of the plurality of data segments, wherein the locality information comprises information regarding a location of the at least one data segment in relation to each other data segment of the plurality of data segments in the backup image;
obtaining an identifier of each data container storing the plurality of data segments of the backup image; and
determining a degree to which the plurality of data segments of the backup image are processed by prefetching.
19. The computing device of claim 18 , wherein the instructions are executable by the processor to:
calculate a prefetch ratio based on a complementary percentage of a value of a predetermined self-read ratio, the prefetch ratio expressing the degree to which the plurality of data segments of the backup image are processed by prefetching.
20. The computing device of claim 18 , wherein the instructions are executable by the processor to:
calculate a data segment sum, wherein calculating the data segment sum comprises summing a number of data segments in the backup image with a number of data segments currently being prefetched by all prefetch threads.