Cloud data rewrites using deduplication
Methods, systems, and devices for data management are described. A data management system (DMS) may identify two snapshots from two different snapshot chains and rewrite one of the snapshots as a cross-incremental snapshot over the other snapshot that includes differential data relative to the other snapshot. Based on rewriting the snapshot as the cross-incremental snapshot, the DMS may reduce the amount of storage space used by the snapshot chain of the cross-incremental snapshot (as cross-incremental snapshots may use less space than base snapshots). In some examples, the DMS may identify which snapshots to rewrite as cross-incremental snapshots based on one or more parameters. For example, the one or more parameters may include respective snapshot expiration times for each of the snapshot chains.
1 . A method, comprising:
generating a plurality of snapshot chains for a target computing object, wherein:
each snapshot chain of the plurality of snapshot chains comprises a respective base snapshot of the target computing object and a plurality of respective incremental snapshots of the target computing object, wherein a respective incremental snapshot of the plurality of respective incremental snapshots in a snapshot chain comprises respective differential data relative to a previous snapshot in the snapshot chain,
a first snapshot chain within the plurality of snapshot chains includes a first base snapshot, and
a second snapshot chain within the plurality of snapshot chains includes a second snapshot, the second snapshot comprising shared data relative to the first base snapshot and further comprising differential data relative to the first base snapshot; and
generating a cross-incremental snapshot, wherein:
the cross-incremental snapshot includes the differential data from the second snapshot relative to the first base snapshot, and
the cross-incremental snapshot replaces the second snapshot in the second snapshot chain.
2 . The method of claim 1 , further comprising:
determining an order of the plurality of snapshot chains in accordance with one or more parameters, wherein:
a position of the first snapshot chain in the order of the plurality of snapshot chains corresponds to a first value of the one or more parameters associated with the first snapshot chain, and
a position of the second snapshot chain in the order of the plurality of snapshot chains corresponds to a second value of the one or more parameters associated with the second snapshot chain.
3 . The method of claim 2 , wherein:
the order of the plurality of snapshot chains is in accordance with respective snapshot chain expiration times for each snapshot chain of the plurality of snapshot chains, the one or more parameters comprising the respective snapshot chain expiration times;
the first value is an earliest snapshot chain expiration time from among the respective snapshot chain expiration times; and
the second value is a second-earliest snapshot chain expiration time from among the respective snapshot chain expiration times.
4 . The method of claim 1 , further comprising:
replacing, within metadata corresponding to a first incremental snapshot of the second snapshot chain, a first reference to a second base snapshot of the second snapshot chain with a second reference to the cross-incremental snapshot based at least in part on generating the cross-incremental snapshot.
5 . The method of claim 1 , further comprising:
generating a second cross-incremental snapshot that includes second differential data from a third snapshot of a third snapshot chain of the plurality of snapshot chains relative to the cross-incremental snapshot, wherein the second cross-incremental snapshot replaces the third snapshot in the third snapshot chain.
6 . The method of claim 1 , further comprising:
reading, from a storage environment, one or more blocks corresponding to the differential data based at least in part on the differential data; and
writing the one or more blocks in metadata corresponding to the cross-incremental snapshot, wherein generating the cross-incremental snapshot is based at least in part on writing the one or more blocks.
7 . The method of claim 1 , further comprising:
storing the plurality of snapshot chains in a storage environment, the plurality of snapshot chains corresponding to a first quantity of storage space in the storage environment, wherein generating the cross-incremental snapshot that replaces the second snapshot within the plurality of snapshot chains results in the plurality of snapshot chains occupying a second quantity of storage space in the storage environment that is less than the first quantity.
8 . The method of claim 1 , further comprising:
deleting, from a storage environment, a first plurality of incremental snapshots in the first snapshot chain based at least in part on an expiration of the first plurality of incremental snapshots, wherein the first base snapshot is retained within the storage environment after deletion of the first plurality of incremental snapshots in the first snapshot chain; and
deleting, from the storage environment, a second plurality of incremental snapshots in the second snapshot chain based at least in part on an expiration of the second plurality of incremental snapshots, wherein a first incremental snapshot of the second plurality of incremental snapshots includes second differential data relative to the cross-incremental snapshot, and wherein the cross-incremental snapshot is retained within the storage environment after deletion of the second plurality of incremental snapshots in the second snapshot chain.
9 . An apparatus, comprising:
one or more memories storing processor-executable code; and
one or more processors coupled with the one or more memories and individually or collectively operable to execute the code to cause the apparatus to:
generate a plurality of snapshot chains for a target computing object, wherein:
each snapshot chain of the plurality of snapshot chains comprises a respective base snapshot of the target computing object and a plurality of respective incremental snapshots of the target computing object, wherein a respective incremental snapshot of the plurality of respective incremental snapshots in a snapshot chain comprises respective differential data relative to a previous snapshot in the snapshot chain,
a first snapshot chain within the plurality of snapshot chains includes a first base snapshot, and
a second snapshot chain within the plurality of snapshot chains includes a second snapshot, the second snapshot comprising shared data relative to the first base snapshot and further comprising differential data relative to the first base snapshot; and
generate a cross-incremental snapshot, wherein:
the cross-incremental snapshot includes the differential data from the second snapshot relative to the first base snapshot, and
the cross-incremental snapshot replaces the second snapshot in the second snapshot chain.
10 . The apparatus of claim 9 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
determine an order of the plurality of snapshot chains in accordance with one or more parameters, wherein:
a position of the first snapshot chain in the order of the plurality of snapshot chains corresponds to a first value of the one or more parameters associated with the first snapshot chain, and
a position of the second snapshot chain in the order of the plurality of snapshot chains corresponds to a second value of the one or more parameters associated with the second snapshot chain.
11 . The apparatus of claim 10 , wherein:
the order of the plurality of snapshot chains is in accordance with respective snapshot chain expiration times for each snapshot chain of the plurality of snapshot chains, the one or more parameters comprising the respective snapshot chain expiration times;
the first value is an earliest snapshot chain expiration time from among the respective snapshot chain expiration times; and
the second value is a second-earliest snapshot chain expiration time from among the respective snapshot chain expiration times.
12 . The apparatus of claim 9 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
replace, within metadata corresponding to a first incremental snapshot of the second snapshot chain, a first reference to a second base snapshot of the second snapshot chain with a second reference to the cross-incremental snapshot based at least in part on generating the cross-incremental snapshot.
13 . The apparatus of claim 9 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
generate a second cross-incremental snapshot that includes second differential data from a third snapshot of a third snapshot chain of the plurality of snapshot chains relative to the cross-incremental snapshot, wherein the second cross-incremental snapshot replaces the third snapshot in the third snapshot chain.
14 . The apparatus of claim 9 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
read, from a storage environment, one or more blocks corresponding to the differential data based at least in part on the differential data; and
write the one or more blocks in metadata corresponding to the cross-incremental snapshot, wherein generating the cross-incremental snapshot is based at least in part on writing the one or more blocks.
15 . The apparatus of claim 9 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
store the plurality of snapshot chains in a storage environment, the plurality of snapshot chains corresponding to a first quantity of storage space in the storage environment, wherein the cross-incremental snapshot that replaces the second snapshot within the plurality of snapshot chains is configured to result in the plurality of snapshot chains occupying a second quantity of storage space in the storage environment that is less than the first quantity.
16 . The apparatus of claim 9 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
delete, from a storage environment, a first plurality of incremental snapshots in the first snapshot chain based at least in part on an expiration of the first plurality of incremental snapshots, wherein the one or more processors are individually or collectively operable to execute the code to cause the apparatus to retain the first base snapshot within the storage environment after deletion of the first plurality of incremental snapshots in the first snapshot chain; and
delete, from the storage environment, a second plurality of incremental snapshots in the second snapshot chain based at least in part on an expiration of the second plurality of incremental snapshots, wherein a first incremental snapshot of the second plurality of incremental snapshots includes second differential data relative to the cross-incremental snapshot, and wherein the one or more processors are individually or collectively operable to execute the code to cause the apparatus to retain the cross-incremental snapshot within the storage environment after deletion of the second plurality of incremental snapshots in the second snapshot chain.
17 . A non-transitory computer-readable medium storing code, the code comprising instructions executable by one or more processors to:
generate a plurality of snapshot chains for a target computing object, wherein:
each snapshot chain of the plurality of snapshot chains comprises a respective base snapshot of the target computing object and a plurality of respective incremental snapshots of the target computing object, wherein a respective incremental snapshot of the plurality of respective incremental snapshots in a snapshot chain comprises respective differential data relative to a previous snapshot in the snapshot chain,
a first snapshot chain within the plurality of snapshot chains includes a first base snapshot, and
a second snapshot chain of the plurality of snapshot chains includes a second snapshot, the second snapshot comprising shared data relative to the first base snapshot and further comprising differential data relative to the first base snapshot; and
generate a cross-incremental snapshot, wherein:
the cross-incremental snapshot includes the differential data from the second snapshot relative to the first base snapshot, and
the cross-incremental snapshot replaces the second snapshot in the second snapshot chain.
18 . The non-transitory computer-readable medium of claim 17 , wherein the instructions are further executable by the one or more processors to:
determine an order of the plurality of snapshot chains in accordance with one or more parameters, wherein:
a position of the first snapshot chain in the order of the plurality of snapshot chains corresponds to a first value of the one or more parameters associated with the first snapshot chain, and
a position of the second snapshot chain in the order of the plurality of snapshot chains corresponds to a second value of the one or more parameters associated with the second snapshot chain.
19 . The non-transitory computer-readable medium of claim 17 , wherein the instructions are further executable by the one or more processors to:
replace, within metadata corresponding to a first incremental snapshot of the second snapshot chain, a first reference to a second base snapshot of the second snapshot chain with a second reference to the cross-incremental snapshot based at least in part on generating the cross-incremental snapshot.
20 . The non-transitory computer-readable medium of claim 17 , wherein the instructions are further executable by the one or more processors to:
generate a second cross-incremental snapshot that includes second differential data from a third snapshot of a third snapshot chain of the plurality of snapshot chains relative to the cross-incremental snapshot, wherein the second cross-incremental snapshot replaces the third snapshot in the third snapshot chain.