Snapshots for distributed object-based datastores

1 . A distributed object-based datastore system comprising:

an object datastore implemented across a plurality of distributed data nodes, the object datastore including a plurality of data objects that are each stored at one or more data blocks at one or more of the plurality of distributed data nodes;

a metadata datastore configured to be instantiable and to store metadata for the plurality of data objects of the object datastore, the metadata including block mapping information that identifies one or more given data blocks for a given data object, wherein a first instance of the metadata datastore includes active block mapping information that is modifiable to identify the one or more given data blocks at which the given data object is stored at a current timepoint; and

a manager node configured to create snapshot images of the object datastore, the manager node implemented by at least one computing device comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, perform operations of the manager node, wherein the operations comprise:

receiving a snapshot request to generate a snapshot image of at least one portion of the object datastore for a specified timepoint, wherein the specified timepoint is a temporal identifier of an instance of the metadata datastore,

instantiating, based on the specified timepoint, a second instance of the metadata datastore using the first instance of the metadata datastore, the second instance preserving a read-only copy of the active block mapping information that was included in the first instance at the specified timepoint, wherein the second instance of the metadata datastore represents the snapshot image for the specified timepoint,

generating a snapshot chain indicating a sequential order of creation for a plurality of snapshot images for the object datastore, wherein the plurality of snapshot images includes the snapshot image, and wherein the snapshot chain indicates, for the snapshot image, (1) a location-agnostic preceding snapshot image from the object datastore and (2) a location-agnostic subsequent snapshot image from the object datastore,

receiving a deletion request to delete the snapshot image,

subsequent to deleting the snapshot image, (1) generating a first set of different object keys based on a first set of object keys associated with the snapshot image and a second set of object keys associated with the location-agnostic preceding snapshot image and (2) generating a second set of different object keys based on the first set of object keys and a third set of object keys associated with the location-agnostic subsequent snapshot image, wherein an object key is a lexicographic identifier that names a particular data object and that is reclaimable upon deletion of the particular data object and a set of data blocks associated with the particular data object, and

generating a set of reclaimable object keys by determining an intersection of the first set of different object keys and the second set of different object keys.

2 . The distributed object-based datastore system of claim 1 , wherein the metadata datastore is a log-structured merge-tree (LSM) data structure in which a modified version of the metadata for the plurality of data objects is recorded as a separate data entry from a data entry recording a previous version of the metadata, and wherein the first instance includes a particular set of data entries that records the active block mapping information.

3 . The distributed object-based datastore system of claim 1 , wherein the manager node includes an active metadata manager and a snapshot metadata manager, and wherein the operations further comprise:

receiving an access request that identifies, via a data path, one or more data objects stored in the object datastore; and

based on determining whether the data path specifies a particular snapshot image of the object datastore, select one of the active metadata manager or the snapshot metadata manager to obtain particular metadata for retrieving the one or more data objects.

4 . The distributed object-based datastore system of claim 1 , wherein at least one of the location-agnostic preceding snapshot image or the location- agnostic subsequent snapshot image also captures the at least one portion of the object datastore.

5 . The distributed object-based datastore system of claim 1 , wherein at least one of the location-agnostic preceding snapshot image or the location-agnostic subsequent snapshot image captures a different portion of the object datastore.

6 . The distributed object-based datastore system of claim 1 , wherein the object datastore is divided into a plurality of buckets each associated with a subset of the plurality of data objects, and wherein the at least one portion of the object datastore includes data objects belonging to one or more buckets of the plurality of buckets.

7 . The distributed object-based datastore system of claim 1 , wherein each data object of the object datastore is identified by a lexicographic key, and wherein the at least one portion of the object datastore includes a set of data objects having respective lexicographic keys with a common prefix indicated by the snapshot request.

8 . The distributed object-based datastore system of claim 1 , further comprising an object deletion service configured to remove certain data objects from the object datastore based on the certain data objects not being identified in the first instance of the metadata datastore or in any snapshot image of the metadata datastore.

9 . The distributed object-based datastore system of claim 8 , wherein the object deletion service is configured to determine whether a particular data object is identified in a particular snapshot image based on searching a respective block mapping information of the particular snapshot image for a unique identifier associated with a lexicographic key assigned to the particular data object.

10 . The distributed object-based datastore system of claim 1 , wherein the second instance of the metadata datastore is instantiated at a plurality of high-availability nodes that uses a consensus protocol to maintain identical copies of the second instance across the plurality of high-availability nodes.

11 . The distributed object-based datastore system of claim 1 , wherein the operations further comprise:

performing a snapshot-comparison operation with two snapshot images of the object datastore in order to perform at least one of: a compaction of the object datastore, or a restore a particular snapshot image.

12 . A distributed computer system comprising:

a plurality of data nodes communicatively coupled with one another and implementing an object-based datastore for a plurality of object keys, wherein an object key is a lexicographic identifier that names a particular data object and that is reclaimable upon deletion of the particular data object and a set of data blocks associated with the particular data object;

a metadata database configured to be instantiable and to store map data for the object-based datastore, the map data locating respective data blocks for a given object key of the object-based datastore; and

one or more manager nodes configured to perform operations comprising:

in response to a first access request that specifies a particular object key for an active timepoint, using a first instance of the metadata database to locate the respective data blocks for the particular data object key at the active timepoint, wherein the first instance is configured to be modifiable to update a respective map data for a present timepoint;

in response to a snapshot request for a snapshot timepoint, wherein the snapshot timepoint is a temporal identifier of a second instance of the metadata database, instantiating, based on the snapshot timepoint, the second instance of the metadata database using the first instance of the metadata database, the second instance preserving a read-only copy of the first instance at the snapshot timepoint, wherein the second instance represents a snapshot image of the object-based datastore for the snapshot timepoint;

in response to a second access request that specifies the particular object key for the snapshot timepoint, using the second instance of the metadata database to locate the respective data blocks that were associated with the particular object key at the snapshot timepoint;

receiving a deletion request to delete the snapshot image;

subsequent to deleting the snapshot image, (1) generating a first set of different object keys based on a first set of object keys associated with the snapshot image and a second set of object keys associated with a preceding snapshot image from a snapshot chain and (2) generating a second set of different object keys based on the first set of object keys and a third set of object keys associated with a subsequent snapshot image from the snapshot chain; and

generating a set of reclaimable object keys by determining an intersection of the first set of different object keys and the second set of different object keys.

13 . The distributed computer system of claim 12 , wherein the metadata database is further configured to include, for each instance of the metadata database, a deleted key table that indicates certain object keys flagged for deletion from the object-based datastore at a respective timepoint, and wherein the operations further comprise:

determining to remove a certain object key from the object-based datastore based on traversing the deleted key table for at least two instances of the metadata database.

14 . The distributed computer system of claim 12 , wherein the second instance of the metadata database is associated with a path mask that identifies a portion of the object-based datastore captured within the snapshot image.

15 . The distributed computer system of claim 14 , wherein the second instance of the metadata database includes:

(i) a first reference for identifying a first preceding snapshot image also associated with the path mask, and

(ii) a second reference for identifying a second preceding snapshot image that immediately precedes the second instance in a snapshot creation sequence.

16 . The distributed computer system of claim 14 , wherein the second access request specifies the snapshot timepoint based on including a path that satisfies the path mask of the second instance of the metadata database.

17 . The distributed computer system of claim 12 , wherein the operations further comprise:

restoring the object-based datastore to a previous state during the snapshot timepoint based on performing a comparison between the first instance of the metadata database and the second instance of the metadata database.

18 . A computer-implemented method comprising:

generating, by one or more computing nodes of a distributed computing platform, a plurality of instances of an instantiable metadata datastore associated with an object-based datastore, wherein (i) a first instance of the instantiable metadata datastore is configured to dynamically identify distributed data blocks that are mapped to object keys of the object-based datastore at a present timepoint, and (ii) one or more other instances of the instantiable metadata datastore each configured to preserve a read-only copy of the first instance at a respective timepoint prior to the present timepoint,

receiving, by the one or more computing nodes, an access request that specifies a particular object key and a particular timepoint, wherein the particular timepoint is a temporal identifier of an instance of the instantiable metadata datastore, and wherein the particular timepoint is different than the present timepoint;

based on the particular timepoint, selecting, by the one or more computing nodes, a particular instance of the one or more other instances of the instantiable metadata datastore that is associated with the particular timepoint;

handling, by the one or more computing nodes, the access request based on using the particular instance of the instantiable metadata datastore to locate the distributed data blocks that were mapped to the particular object key during the particular timepoint;

receiving a deletion request to delete the particular instance of the instantiable metadata datastore;

subsequent to deleting the particular instance of the instantiable metadata datastore, (1) generating a first set of different object keys based on a first set of object keys associated with the particular instance of the instantiable metadata datastore and a second set of object keys associated with a preceding instance of the instantiable metadata datastore and (2) generating a second set of different object keys based on the first set of object keys and a third set of object keys associated with a subsequent instance of the instantiable metadata datastore; and

generating a set of reclaimable object keys by determining an intersection of the first set of different object keys and the second set of different object keys.

19 . The distributed computer system of claim 12 , wherein the snapshot chain indicates a sequential order of creation for a plurality of snapshot images for the object-based datastore, wherein the plurality of snapshot images includes the snapshot image, wherein the preceding snapshot image is a location-agnostic preceding snapshot image from the object-based datastore, and wherein the subsequent snapshot image is a location-agnostic subsequent snapshot image from the object-based datastore.

20 . The computer-implemented method of claim 18 , wherein the particular instance of the instantiable metadata datastore, the preceding instance of the instantiable metadata datastore, and the subsequent instance of the instantiable metadata datastore are included in a snapshot chain indicating a location-agnostic sequential order of creation for the plurality of instances of the instantiable metadata datastore.