Systems and methods for a snapshot of data
View Patent ↗In one embodiment, a user or client device is connected to a distributed file system comprised of one or more physical nodes. The data on each of the physical nodes store metadata about files and directories within the file system. Some of the embodiments permit a user to take a snapshot of data stored on the file system. The snapshot may include a single file, a single directory, a plurality of files within a directory, a plurality of directories, a path on the file system that includes nested files and subdirectories, or more than one path on the file system that each includes nested files and directories. In one embodiment, previous versions of a file or directory are stored only in response to a request to modify the file or directory. The system creates and updates a snapshot tracking data structure which provides information regarding each file or directory governed by the snapshot that was modified or deleted after the snapshot was taken. The snapshot tracking data structure may be used by the system to perform functions related to the snapshot, including deleting the snapshot.
1. A data storage system accessible by a processor comprising:
at least one storage device;
at least one computer processor;
a hierarchical data storage structure stored in at least one storage device and accessible by the at least one computer processor, the hierarchical data storage structure comprising metadata in a tree structure, each metadata associated with a file or directory;
a snapshot identifier associated with a snapshot, of the snapshot comprising members as of a past time, the members comprising a file or directory and descendants of the file or directory, the snapshot identifier stored in at least in a metadata associated with a file or directory;
a snapshot tracking data structure comprising the snapshot identifier and configured to store references to metadata associated with snapshot versions of modified members, the modified members comprising members which have been modified or deleted after the past time; and
at least one executable software module executed by the at least one computer processor and configured to add a reference to the snapshot tracking data structure, the reference referring to a metadata associated with a snapshot version of a deleted member, the deleted member comprising a member which has been deleted after the past time.
2. The data storage system of claim 1 , wherein the at least one executable software module is further configures to:
receive a request to delete the snapshot; and
if a request to delete the snapshot is received, for each member which has been modified or deleted after the past time,
determine whether the member is governed by a earlier snapshot,
if the member is governed by an earlier snapshot,
access a metadata associated with a earlier snapshot version of the member, the accessed metadata comprising zero or more place-holder records, and
for each of the zero or more place-holder records, replace the place-holder record with a reference to at least a portion of the snapshot version of the member, and
delete a metadata associated with the snapshot version of the member.
3. The data storage system of claim 1 , further comprising:
a index table comprising entries for each snapshot version of each of the members, each entry comprising an identifier of a metadata associated with the member, a snapshot identifier, and a reference to a snapshot metadata associated with the snapshot version of the member;
wherein the at least one executable software module is further configured to access different versions of each of the members by retrieving one or more of the entries which comprise an identifier of a metadata associated with the member.
4. The data storage system of claim 1 , wherein the at least one executable software module is further configured to use the references in the snapshot tracking data structure to track disk usage by the snapshot.
5. The data storage system of claim 1 , wherein the snapshot identifier is based on a global count of the data storage system at a time when the snapshot was taken, the global count indicating a time relative to when s snapshot is taken.
6. The data storage system of claim 1 , further comprising a variable number of snapshot tracking data structures, wherein the snapshot tracking data structure is one of the variable number of snapshot tracking data structures, each of the plurality of snapshot tracking data structures associated with one of a variable numbers of snapshots.
7. The data storage system of claim 1 , wherein the file or directory is not a root directory of the system.
8. A method for managing snapshots in a hierarchical storage system comprising:
obtaining, by a computer processor, a snapshot identifier associated with a snapshot, the snapshot comprising members as of a past time, the members comprising a file or directory and descendants of the file or directory;
storing by the computer processor, the snapshot identifier at least in a metadata associated with the file or directory;
creating, by the computer processor, a snapshot tracking data structure comprising the snapshot identifier and configured to store references to metadata associated with snapshot versions of modified members, the modified members comprising members which have been modified or deleted after the past time; and
adding a reference to the snapshot tracking data structure if a member is deleted after the past time, the reference referring to a metadata associated with a snapshot version of the deleted member.
9. The method of claim 8 , further comprising:
receiving a request to delete the snapshot; and
if a request to delete the snapshot is received, for each member which has been modified or deleted after the past time,
determining whether the member is governed by a earlier snapshot,
if the member is governed by a earlier snapshot,
accessing a metadata associated with a earlier snapshot version of the member, the accessed metadata comprising zero or more place-holder records, and
for each of the zero or more place-holder records, replacing the place-holder record with a reference to a least of portion of the snapshot version of the member, and
deleting a metadata associated with the snapshot version of the member.
10. The method of claim 8 , further comprising:
accessing different versions of each of the members by retrieving one or more entries, which comprise an identifier of a metadata associated with the member, from an index table, the index table comprising entries for each snapshot version of each of the members, each entry comprising an identifier of a metadata associated with the member, a snapshot identifier, and a reference to a snapshot metadata associated with the snapshot version of the member.
11. The method of claim 8 , further comprising using the references in the snapshot tracking data structure to track disk usage by the snapshot.
12. The method of claim 8 , wherein the snapshot identifier is based on a global count of the hierarchical storage system at a time when the snapshot was taken, the global count indicating a time relative to when a snapshot is taken.
13. The method of claim 8 , wherein the file or directory is not a root directory of the system.