HDAG backup system with variable retention
View Patent ↗A storage system efficiently allocates storage space for data backup. An illustrative storage system comprises a storage and backup logic that manages data backup in the storage and converts a file to a hash-based directed acyclic graph (HDAG) so that a hierarchy of nodes of the HDAGs contain a hash of child nodes. The backup logic creates multiple HDAGs for one or more of a plurality of retention policies and represents directories by a first node containing a payload of metadata and weak references to roots of child nodes, and a second node containing references to the first node and to any HDAGs corresponding to files and/or directories governed by an active policy of the plurality of retention policies.
1. A storage system comprising:
a storage; and
backup logic that manages data backup in the storage, the backup logic creates multiple hash-directed acyclic graphs (HDAGs) for ones of a plurality of retention policies,
wherein one of the multiple HDAGs represents directories by a first node and a second node to be backed up, and the backup logic performs backup operations using the multiple HDAGs to attain the plurality of retention policies.
2. The storage system according to claim 1 , wherein the second node contains references to the first node and contains references to any of the multiple HDAGs corresponding to files or directories governed by a currently active policy of the plurality of retention policies.
3. The storage system according to claim 1 further comprising:
the backup logic to store a backup and associate nodes of one of the multiple HDAGs with a retention group in a multiple-pass process operating upon the retention group and a working set, each of which is individually identifiable, the working set to support more than one retention group and having an individually-associated retention policy.
4. The storage system according to claim 1 wherein:
the backup logic to store retention groups and associate queries as made and nodes as stored with a working set wherein the nodes contained in the working set are not collected as long as the working set is valid.
5. The storage system according to claim 1 ,
wherein the backup logic performs a backup operation comprising sequencing through at least part of a directory tree, creating the multiple HDAGs, one for each of the plurality of retention policies; and operates subsequent to backup comprising identifying at least one retention group, each associated with one or more HDAG roots and associated with a retention policy of the plurality of retention policies specifying duration of HDAG retention, and further comprising requesting the storage to construct retention groups, identifying in the request retention policy, roots, and group-identification metadata.
6. The storage system according to claim 1 ,
wherein the backup logic maintains a chronology encapsulating a record for a plurality of related unexpired retention groups in combination with associated ones of the plurality of retention policies and metadata, the chronology comprising a plurality of classes that can be at least one of modified and deleted in response to assertion of backup against the chronology, the classes containing a list of unexpired retention groups and expirations.
7. The method of claim 1 , wherein the first node contains a payload of metadata and references to roots of child nodes.
8. The method of claim 7 , wherein the references to roots of child nodes allow contents to be obtained if existing in the storage but does not prevent the contents from expiring from storage.
9. A data processing method comprising:
managing data backup comprising:
creating multiple hash-directed acyclic graphs (HDAGs) for ones of a plurality of retention policies, wherein one of the multiple HDAGs represents directories by a first node and a second node to be backed up;
and
executing backup operations using the multiple HDAGs to attain the plurality of retention policies.
10. The method according to claim 9 , wherein the second node contains references to the first node and contains references to any of the multiple HDAGs corresponding to files or directories governed by an active policy of the plurality of retention policies.
11. The method according to claim 9 further comprising:
storing a backup;
associating nodes of one of the multiple HDAGs with a retention group in a multiple-pass process operating upon the retention group and a working set, each of which is individually identifiable;
using the working set to support more than one retention group and having an individually-associated retention policy; and
storing retention groups and associating queries as made and nodes as stored with the working set wherein the storage guarantees that as long as the working set is valid, the nodes contained in the working set are not collected.
12. The method according to claim 9 further comprising:
performing a backup operation comprising:
sequencing through at least part of a directory tree; and
creating the multiple HDAGs, one for each of the plurality of retention policies; and
operating subsequent to backup comprising:
identifying at least one retention group, each associated with one or more HDAG roots and associated with a retention policy of the plurality of retention policies specifying duration of HDAG retention;
requesting the storage to construct retention groups; and
identifying in the request retention policy, roots, and group identification metadata.
13. The storage system of claim 9 , wherein the first node contains a payload of metadata and references to roots of child nodes.
14. The storage system of claim 13 , wherein the references to roots of child nodes allow contents to be obtained if existing in the storage but does not prevent the contents from expiring from the storage.