Fractional consistent global snapshots of a distributed namespace
A method of capturing a fractional snapshot of a distributed file system includes receiving a snapshot request to generate a factional snapshot of a file system. The request includes an indication of a subset of data of the file system. The method further includes identifying, by a processing device, the subset of data of the file system indicated by the snapshot request and generating the fractional snapshot comprising the subset of data of the file system indicated by the snapshot request.
1. A method of capturing a fractional snapshot of a distributed file system, comprising:
receiving a snapshot request to generate a fractional snapshot of a file system, the snapshot request comprising an indication of a subset of data of the file system, wherein the indication in the snapshot request comprises a list of storage objects, wherein each storage object includes a collection partition identifying a shadow M-Tree of the file system and a storage location of the shadow M-Tree;
determining whether a best-efforts flag is set for the fractional snapshot of the file system;
identifying, by a processing device, the subset of data of the file system indicated by the snapshot request in response to determining that the best-efforts flag is set; and
generating, by the processing device, the fractional snapshot comprising the subset of data of the file system indicated by the snapshot request.
2. The method of claim 1 , wherein the file system comprises one or more file sets and wherein the fractional snapshot comprises a subset of the one or more file sets.
3. The method of claim 2 , wherein the indication of the subset of data comprises a list of storage objects, each storage object identifying a data node and storage location of a file set to be included in the fractional snapshot.
4. The method of claim 2 , further comprising:
receiving a request to remove the fractional snapshot;
determining the file sets included in the fractional snapshot; and
removing the fractional snapshot for each of the file sets of the fractional snapshot.
5. The method of claim 2 , wherein generating the fractional snapshot comprises:
determining whether each of the file sets are available; and
generating the fractional snapshot with the file sets that are available.
6. The method of claim 5 , wherein generating the fractional snapshot comprises:
determining whether each of the file sets have been modified since a last snapshot; and
generating the fractional snapshot on file sets that have been modified.
7. The method of claim 1 , wherein the file system comprises a meta node and one or more data nodes, and wherein the fractional snapshot is generated for a subset of the one or more data nodes.
8. A system comprising:
a memory; and
a processing device operatively coupled to the memory, the processing device to perform operations comprising:
receiving a snapshot request to generate a fractional snapshot of a file system, the snapshot request comprising an indication of a subset of data of the file system, wherein the indication in the snapshot request comprises a list of storage objects, wherein each storage object includes a collection partition identifying a shadow M-Tree of the file system and a storage location of the shadow M-Tree;
determining whether a best-efforts flag is set for the fractional snapshot of the file system;
identifying the subset of data of the file system indicated by the snapshot request in response to determining that the best-efforts flag is set; and
generating the fractional snapshot comprising the subset of data of the file system indicated by the snapshot request.
9. The system of claim 8 , wherein the file system comprises one or more file sets and wherein the fractional snapshot comprises a subset of the one or more file sets.
10. The system of claim 9 , wherein the indication of the subset of data comprises a list of storage objects, each storage object identifying a data node and storage location of a file set to be included in the fractional snapshot.
11. The system of claim 9 , further comprising:
receiving a request to remove the fractional snapshot;
determining the file sets included in the fractional snapshot; and
removing the fractional snapshot for each of the file sets of the fractional snapshot.
12. The system of claim 9 , wherein generating the fractional snapshot comprises:
determining whether each of the file sets are available; and
generating the fractional snapshot with the file sets that are available.
13. The system of claim 12 , wherein generating the fractional snapshot comprises:
determining whether each of the file sets have been modified since a last snapshot; and
generating the fractional snapshot on file sets that have been modified.
14. The system of claim 8 , wherein the file system comprises a meta node and one or more data nodes, and wherein the fractional snapshot is generated for a subset of the one or more data nodes.
15. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform one or more operations, the operations comprising:
receiving a snapshot request to generate a fractional snapshot of a file system, the snapshot request comprising an indication of a subset of data of the file system, wherein the indication in the snapshot request comprises a list of storage objects, wherein each storage object includes a collection partition identifying a shadow M-Tree of the file system and a storage location of the shadow M-Tree;
determining whether a best-efforts flag is set for the fractional snapshot of the file system;
identifying, by a processing device, the subset of data of the file system indicated by the snapshot request in response to determining that the best-efforts flag is set; and
generating, by the processing device, the fractional snapshot comprising the subset of data of the file system indicated by the snapshot request.
16. The non-transitory machine-readable medium of claim 15 , wherein the file system comprises one or more file sets and wherein the fractional snapshot comprises a subset of the one or more file sets.
17. The non-transitory machine-readable medium of claim 16 , wherein the indication of the subset of data comprises a list of storage objects, each storage object identifying a data node and storage location of a file set to be included in the fractional snapshot.
18. The non-transitory machine-readable medium of claim 16 , further comprising:
receiving a request to remove the fractional snapshot;
determining the file sets included in the fractional snapshot; and
removing the fractional snapshot for each of the file sets of the fractional snapshot.
19. The non-transitory machine-readable medium of claim 16 , wherein generating the fractional snapshot comprises:
determining whether each of the file sets are available; and
generating the fractional snapshot with the file sets that are available.
20. The non-transitory machine-readable medium of claim 19 , wherein generating the fractional snapshot comprises:
determining whether each of the file sets have been modified since a last snapshot; and
generating the fractional snapshot on file sets that have been modified.