Lightweight consolidation operation for storage space reclamation
Methods, systems, and devices for data management are described. A data management system (DMS) may perform a lightweight consolidation operation on an expired snapshot rather than a full consolidation operation that merges the expired snapshot into another snapshot based on one or more snapshot heuristics. The lightweight consolidation operation may delete data that is not shared with (e.g., referenced by, relied upon by) any subsequent incremental snapshot without reading all of the data within the snapshot and without deleting the expired snapshot or writing data into a new file. The DMS may perform a full consolidation operation if a quantity of leaked space satisfies a threshold. Additionally, or alternatively, the DMS may perform the full consolidation operation if a snapshot chain length exceeds a threshold and if performing the full consolidation operation would reduce the used space by a threshold quantity.
1 . A method, comprising:
generating a plurality of snapshots of a target computing object, each snapshot of the plurality of snapshots comprising data stored across one or more respective data stripes;
identifying an expiration of a first snapshot from among the plurality of snapshots, the first snapshot corresponding to a first state of the target computing object as of a first point in time, and the first snapshot comprising data stored across one or more first data stripes;
reading, based at least in part on the expiration of the first snapshot, metadata corresponding to the first snapshot, the metadata indicating that at least one first data stripe from among the one or more first data stripes was entirely overwritten by a second snapshot corresponding to a second state of the target computing object as of a second point in time;
performing, based at least in part on the at least one first data stripe being entirely overwritten by the second snapshot, a consolidation operation that deletes the at least one first data stripe that was entirely overwritten by the second snapshot while retaining at least one other first data stripe that was not entirely overwritten by the second snapshot;
determining a quantity of leaked space, the quantity of leaked space associated with a ratio of a quantity of available first data stripes of the one or more first data stripes to a total quantity of the one or more first data stripes; and
performing, based at least in part on the quantity of leaked space satisfying a threshold, a second consolidation operation that generates a third snapshot that includes the data in the first snapshot and data in the second snapshot and deletes the first snapshot and the second snapshot.
2 . The method of claim 1 , wherein the threshold is based at least in part on a quantity of available storage space in a storage environment for the plurality of snapshots.
3 . The method of claim 2 , further comprising:
adjusting the threshold based at least in part on the quantity of available storage space satisfying a second threshold.
4 . The method of claim 1 , further comprising:
generating one or more snapshot chains based at least in part on generating the plurality of snapshots, a respective snapshot chain comprising a base snapshot and one or more incremental snapshots that include differential data relative to a previous snapshot in the respective snapshot chain, wherein a first snapshot chain comprises the first snapshot and the second snapshot; and
performing, based at least in part on a chain length of the first snapshot chain satisfying a threshold length, a third consolidation operation that generates a fourth snapshot that includes the data in the first snapshot and the data in the second snapshot and deletes the first snapshot and the second snapshot.
5 . The method of claim 4 , wherein performance of the second consolidation operation is further based at least in part on a threshold quantity of available storage space in a storage environment for the plurality of snapshots.
6 . The method of claim 1 , further comprising:
adding, as part of the consolidation operation, one or more markers to the metadata indicating to delete the at least one first data stripe that was entirely overwritten based at least in part on reading the metadata;
scanning, in accordance with a periodicity, the metadata for the one or more markers; and
deleting the at least one first data stripe based at least in part on scanning the metadata and detecting the one or more markers.
7 . An apparatus, comprising:
one or more memories storing processor-executable code; and
one or more processors coupled with the one or more memories and individually or collectively operable to execute the code to cause the apparatus to:
generate a plurality of snapshots of a target computing object, each snapshot of the plurality of snapshots comprising data stored across one or more respective data stripes;
identify an expiration of a first snapshot from among the plurality of snapshots, the first snapshot corresponding to a first state of the target computing object as of a first point in time, and the first snapshot comprising data stored across one or more first data stripes;
read, based at least in part on the expiration of the first snapshot, metadata corresponding to the first snapshot, the metadata indicating that at least one first data stripe from among the one or more first data stripes was entirely overwritten by a second snapshot corresponding to a second state of the target computing object as of a second point in time;
perform, based at least in part on the at least one first data stripe being entirely overwritten by the second snapshot, a consolidation operation that deletes the at least one first data stripe that was entirely overwritten by the second snapshot while retaining at least one other first data stripe that was not entirely overwritten by the second snapshot;
determine a quantity of leaked space, the quantity of leaked space associated with a ratio of a quantity of available first data stripes of the one or more first data stripes to a total quantity of the one or more first data stripes; and
perform, based at least in part on the quantity of leaked space satisfying a threshold, a second consolidation operation that generates a third snapshot that includes the data in the first snapshot and data in the second snapshot and deletes the first snapshot and the second snapshot.
8 . The apparatus of claim 7 , wherein the threshold is based at least in part on a quantity of available storage space in a storage environment for the plurality of snapshots.
9 . The apparatus of claim 8 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
adjust the threshold based at least in part on the quantity of available storage space satisfying a second threshold.
10 . The apparatus of claim 7 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
generate one or more snapshot chains based at least in part on generating the plurality of snapshots, a respective snapshot chain comprising a base snapshot and one or more incremental snapshots that include differential data relative to a previous snapshot in the respective snapshot chain, wherein a first snapshot chain comprises the first snapshot and the second snapshot; and
perform, based at least in part on a chain length of the first snapshot chain satisfying a threshold length, a third consolidation operation that generates a fourth snapshot that includes the data in the first snapshot and the data in the second snapshot and deletes the first snapshot and the second snapshot.
11 . The apparatus of claim 10 , wherein performance of the second consolidation operation is further based at least in part on a threshold quantity of available storage space in a storage environment for the plurality of snapshots.
12 . The apparatus of claim 7 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
add, as part of the consolidation operation, one or more markers to the metadata indicating to delete the at least one first data stripe that was entirely overwritten based at least in part on reading the metadata;
scan, in accordance with a periodicity, the metadata for the one or more markers; and
delete the at least one first data stripe based at least in part on scanning the metadata and detecting the one or more markers.
13 . A non-transitory computer-readable medium storing code, the code comprising instructions executable by one or more processors to:
generate a plurality of snapshots of a target computing object, each snapshot of the plurality of snapshots comprising data stored across one or more respective data stripes;
identify an expiration of a first snapshot from among the plurality of snapshots, the first snapshot corresponding to a first state of the target computing object as of a first point in time, and the first snapshot comprising data stored across one or more first data stripes;
read, based at least in part on the expiration of the first snapshot, metadata corresponding to the first snapshot, the metadata indicating that at least one first data stripe from among the one or more first data stripes was entirely overwritten by a second snapshot corresponding to a second state of the target computing object as of a second point in time;
perform, based at least in part on the at least one first data stripe being entirely overwritten by the second snapshot, a consolidation operation that deletes the at least one first data stripe that was entirely overwritten by the second snapshot while retaining at least one other first data stripe that was not entirely overwritten by the second snapshot;
determine a quantity of leaked space, the quantity of leaked space associated with a ratio of a quantity of available first data stripes of the one or more first data stripes to a total quantity of the one or more first data stripes; and
perform, based at least in part on the quantity of leaked space satisfying a threshold, a second consolidation operation that generates a third snapshot that includes the data in the first snapshot and data in the second snapshot and deletes the first snapshot and the second snapshot.
14 . The non-transitory computer-readable medium of claim 13 , wherein the threshold is based at least in part on a quantity of available storage space in a storage environment for the plurality of snapshots.
15 . The non-transitory computer-readable medium of claim 14 , wherein the instructions are further executable by the one or more processors to:
adjust the threshold based at least in part on the quantity of available storage space satisfying a second threshold.
16 . The non-transitory computer-readable medium of claim 13 , wherein the instructions are further executable by the one or more processors to:
generate one or more snapshot chains based at least in part on generating the plurality of snapshots, a respective snapshot chain comprising a base snapshot and one or more incremental snapshots that include differential data relative to a previous snapshot in the respective snapshot chain, wherein a first snapshot chain comprises the first snapshot and the second snapshot; and
perform, based at least in part on a chain length of the first snapshot chain satisfying a threshold length, a third consolidation operation that generates a fourth snapshot that includes the data in the first snapshot and the data in the second snapshot and deletes the first snapshot and the second snapshot.
17 . The non-transitory computer-readable medium of claim 13 , wherein the instructions are further executable by the one or more processors to:
add, as part of the consolidation operation, one or more markers to the metadata indicating to delete the at least one first data stripe that was entirely overwritten based at least in part on reading the metadata;
scan, in accordance with a periodicity, the metadata for the one or more markers; and
delete the at least one first data stripe based at least in part on scanning the metadata and detecting the one or more markers.