Global snapshot backups of a distributed name space
Embodiments for enabling snapshot backups in a global name space of a cluster network, by representing the name space of cluster network in an MTree, storing data files organized in a B+ Tree format on one or more data nodes, storing name specific information of the data files in a B+ Tree format in a meta node, wherein a B+ Tree of the meta node accesses each corresponding B+ Tree in each of the one or more data nodes. The process takes snapshot backups of individual MTree limbs, and links the limbs of each snapshot into groups based on a cluster identifier and snapshot identifier.
1. A method enabling snapshot backups in a global name space of a cluster network, comprising:
representing the name space of cluster network in an MTree;
storing data files organized in a hierarchical tree format on one or more data nodes;
storing name specific information of the data files in a B+ Tree format in a meta node, wherein a B+ Tree of the meta node accesses each corresponding B+ Tree in each of the one or more data nodes;
taking snapshot backups of individual MTree limbs, each individual MTree limb representing an individual node within the cluster network;
linking the limbs of each snapshot into groups based on a cluster ID and snapshot ID;
forming, by the meta node, a global snapshot from the snapshot backups of the individual MTree limbs;
broadcasting a remote procedure call to all limb nodes of the MTree to initiate a cluster-wide quiescing process;
flushing, by each node, data content in transit and stabilizing a block map of all dirty files;
persisting each files block map inode record into a local data space B+ Tree; and
registering and storing a cluster-wide identity of the global snapshot.
2. The method of claim 1 wherein the name specific information comprise metadata specifying at least one of the following: file name, file ownership, permissions, file properties, and attributes.
3. The method of claim 1 wherein the meta node generates an MTree unique identifier for use as a cluster-wide snapshot identifier.
4. The method of claim 1 further comprising organizing the B+ Trees of each of the meta and data nodes into a higher level UTree to form a hierarchical tree structure.
5. The method of claim 4 further comprising configuring each node with a respective UTree.
6. The method of claim 1 wherein files in each B+ Tree all reference the same pool of unique segments made up of log-structured containers that organize the segments on disk to optimize throughput and deduplication backup effectiveness.
7. The method of claim 1 wherein the cluster network comprises a Data Domain File System (DDFS) deduplication backup system.
8. A system for enabling snapshot backups in a global name space of a cluster network, the system comprising:
a first component representing the name space of cluster network in an MTree;
a storage device storing data files organized in a hierarchical tree format on one or more data nodes, and storing name specific information of the data files in a B+ Tree format in a meta node, wherein a B+ Tree of the meta node accesses each corresponding B+ Tree in each of the one or more data nodes;
a backup component taking snapshot backups of individual MTree limbs, each individual MTree limb representing an individual node within the cluster network, and linking the limbs of each snapshot into groups based on a cluster ID and snapshot ID, wherein a global snapshot is formed by the meta node from the snapshot backups of the individual MTree limbs;
an interface component broadcasting a remote procedure call to all limb nodes of the MTree to initiate a cluster-wide quiescing process; and
a second component flushing, by each node, data content in transit and stabilizing a block map of all dirty files, persisting each files block map inode record into a local data space B+ Tree, and registering and storing a cluster-wide identity of the global snapshot.
9. The system of claim 8 wherein the name specific information comprise metadata specifying at least one of the following: file name, file ownership, permissions, file properties, and attributes.
10. The system of claim 8 wherein the meta node generates an MTree unique identifier for use as a cluster-wide snapshot identifier.
11. The system of claim 8 wherein the B+ Trees of each of the meta and data nodes are organized into a higher level UTree to form a hierarchical tree structure.
12. The system of claim 11 wherein each node is configured with a respective UTree.
13. The system of claim 8 wherein the cluster network comprises a Data Domain File System (DDFS) deduplication backup system.
14. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method for enabling snapshot backups in a global name space of a cluster network, the method comprising:
representing the name space of cluster network in an MTree;
storing data files organized in a hierarchical tree format on one or more data nodes;
storing name specific information of the data files in a B+ Tree format in a meta node, wherein a B+ Tree of the meta node accesses each corresponding B+ Tree in each of the one or more data nodes;
taking snapshot backups of individual MTree limbs, each individual MTree limb representing an individual node within the cluster network;
linking the limbs of each snapshot into groups based on a cluster ID and snapshot ID;
forming, by the meta node, a global snapshot from the snapshot backups of the individual MTree limbs;
broadcasting a remote procedure call to all limb nodes of the MTree to initiate a cluster-wide quiescing process;
flushing, by each node, data content in transit and stabilizing a block map of all dirty files;
persisting each files block map inode record into a local data space B+ Tree; and
registering and storing a cluster-wide identity of the global snapshot.