Methods and systems for affinity aware container prefetching
Disclosed are techniques that provide for deduplication in an efficient and effective manner. For example, such methods, computer program products, and computer systems can include generating new feature information for one or more portions of a new backup image, generating first container range information by performing a container range calculation using the new feature information, generating existing feature information for one or more portions of an existing backup image, generating second container range information by performing the container range calculation using the existing feature information, determining a container range affinity between the first container range information and the second container range information, identifying at least one portion of the one or more portions of the existing backup image using a result of the determining, and prefetching the one or more fingerprints corresponding to the at least one portion of the one or more portions of the existing backup image.
1. A method comprising:
determining first container range information for a first one or more containers of a first plurality of containers of a new backup image, wherein
the determining the first container range information comprises
generating new feature information from the first one or more containers, and
generating the first container range information from the new feature information, and
the generating the first container range information comprises performing a container range calculation;
determining a container range affinity between the first container range information and second container range information, wherein
the second container range information is for a second one or more containers of a second plurality of containers of one or more existing backup images;
identifying at least one container of the second one or more containers based, at least in part, on a result of the determining the container range affinity;
prefetching the at least one container; and
performing a deduplication operation on a unit of data of the new backup image, wherein
the deduplication operation comprises
comparing
a fingerprint of the unit of data
and
one or more fingerprints of one or more units of data of the at least one container.
2. The method of claim 1 , wherein
the new feature information was generated as an output of a machine learning process that received the first one or more containers as an input, and
the generating the second container range information comprises
performing the container range calculation using existing feature information for the one or more existing backup images, wherein
the existing feature information was generated as an output of a machine learning process that received the second one or more containers as another input.
3. The method of claim 1 , wherein
the new feature information is generated as a first output of a machine learning process, and
the machine learning process receives the first one or more containers as an input.
4. The method of claim 3 , further comprising:
generating existing feature information for the second one or more containers, wherein
the existing feature information is generated as a second output of the machine learning process,
the machine learning process receives the second one or more containers as another input and
the generating the second container range information comprises performing the container range calculation using the existing feature information.
5. The method of claim 1 , further comprising:
storing the second container range information, wherein
the storing facilitates determining another container range affinity between third container range information and the second container range information, and
the third container range information is for one or more portions of another new backup image.
6. The method of claim 1 , further comprising:
maintaining affinity information, wherein
the affinity information facilitates the determining by virtue of being configured to represent a frequency of containers of a plurality of existing backup images selected based on one or more container ranges of a plurality of new backup images.
7. The method of claim 1 , wherein
the determining is performed by a container range determiner,
an output of the container range determiner is container range affinity information, and
the container range affinity information is stored in a container range information data structure.
8. The method of claim 7 , wherein
the container range affinity information comprises
a container range, and
an affinity score, and
the container range comprises
lower bound,
mean, and
upper bound.
9. The method of claim 1 , wherein
the first container range information is determined using a first range control value,
the second container range information is determined using a second range control value, and
the first range control value is larger than the second range control value.
10. A non-transitory computer-readable storage medium, comprising program instructions, which, when executed by one or more processors of a computing system, perform a method comprising:
determining first container range information for a first one or more containers of a first plurality of containers of a new backup image, wherein
the determining the first container range information comprises
generating new feature information from the first one or more containers, and
generating the first container range information from the new feature information, and
the generating the first container range information comprises performing a container range calculation;
determining a container range affinity between the first container range information and second container range information, wherein
the second container range information is for a second one or more containers of a second plurality of containers of one or more existing backup images;
identifying at least one container of the second one or more containers based, at least in part, on a result of the determining the container range affinity;
prefetching the at least one container; and
performing a deduplication operation on a unit of data of the new backup image, wherein
the deduplication operation comprises
comparing
a fingerprint of the unit of data
and
one or more fingerprints of one or more units of data of the at least one container.
11. The non-transitory computer-readable storage medium of claim 10 , wherein
the new feature information is generated as a first output of a machine learning process, and
the machine learning process receives the first one or more containers.
12. The non-transitory computer-readable storage medium of claim 11 , the method further comprising:
generating existing feature information for the second one or more containers portions of the existing backup image, wherein
the existing feature information is generated as a second output of the machine learning process,
the machine learning process receives the second one or more containers portions of the existing backup image as another input and
the generating the second container range information comprises performing the container range calculation using the existing feature information.
13. The non-transitory computer-readable storage medium of claim 10 , the method further comprising:
storing the second container range information, wherein
the storing facilitates determining another container range affinity between third container range information and the second container range information, and
the third container range information is for one or more portions of another new backup image.
14. The non-transitory computer-readable storage medium of claim 10 , the method further comprising:
maintaining affinity information, wherein
the affinity information facilitates the determining by virtue of being configured to represent a frequency of containers of a plurality of existing backup images selected based on one or more container ranges of a plurality of new backup images.
15. The non-transitory computer-readable storage medium of claim 10 , wherein
the determining is performed by a container range determiner,
an output of the container range determiner is container range affinity information, and
the container range affinity information is stored in a container range information data structure.
16. The non-transitory computer-readable storage medium of claim 15 , wherein
the container range affinity information comprises
a container range, and
an affinity score, and
the container range comprises
lower bound,
mean, and
upper bound.
17. The non-transitory computer-readable storage medium of claim 10 , wherein
the first container range information is determined using a first range control value,
the second container range information is determined using a second range control value, and
the first range control value is larger than the second range control value.
18. A computing system comprising:
one or more processors; and
a computer-readable storage medium coupled to the one or more processors, comprising program instructions, which, when executed by the one or more processors, perform a method comprising
determining first container range information for a first one or more containers of a first plurality of containers of a new backup image, wherein
the determining the first container range information comprises
generating new feature information from the first one or more containers, and
generating the first container range information from the new feature information, and
the generating the first container range information comprises performing a container range calculation,
determining a container range affinity between the first container range information and second container range information, wherein
the second container range information is for a second one or more containers of a second plurality of containers of one or more existing backup images,
identifying at least one portion of the one or more portions of the one or more existing backup images based, at least in part, on a result of the determining,
identifying at least one container of the second one or more containers based, at least in part, on a result of the determining the container range affinity,
prefetching the at least one container, and
performing a deduplication operation on a unit of data of the new backup image, wherein
the deduplication operation comprises
comparing
a fingerprint of the unit of data
and
one or more fingerprints of one or more units of data of the at least one container.
19. The computing system of claim 18 , wherein
the new feature information is generated as a first output of a machine learning process, the machine learning process receives the first one or more containers as an input,
the method further comprises
generating existing feature information for the second one or more containers, wherein
the existing feature information is generated as a second output of the machine learning process,
the machine learning process receives the second one or more containers as another input s and
the generating the second container range information comprises performing the container range calculation using the existing feature information.
20. The method of claim 1 , wherein
the one or more fingerprints are stored in at least one of
the at least one container, or
a fingerprint catalog.