Compliance namespace separation to achieve snapshot based consistency across failover-failback while maintaining data retention regulation compliance
Embodiments of the present disclosure relate to creating and maintaining a separate compliance store within a domain. Files that are committed within the domain are linked to the compliance store using canonical links. Canonical links are guaranteed to be non-erasable and non-rewritable for the duration of their retention period. When using a source cluster of nodes as a primary production cluster and a target cluster of nodes as a backup cluster, file data and canonical links can be synced from the source cluster to the target cluster to maintain compliance with data retention rules. Non-canonical links can be modified as necessary to ensure snapshot consistency when failover or failback occurs between a source cluster of nodes and a target cluster of nodes.
1. A method comprising:
maintaining on a source cluster of computing nodes a source domain that includes a source compliance store;
maintaining on a target cluster of the computing nodes a target domain that includes a target compliance store; and
establishing a new sync between the source cluster of the computing nodes to the target cluster of the computing nodes wherein the target cluster operates as a failover target, and wherein the syncing comprises at least:
delaying an application of commits by creating a new target cluster side system b-tree to track which files are to be committed during the new sync;
sending pending canonical links during the new sync, wherein each pending canonical links reference a write once read many (WORM) file, wherein each canonical link is guaranteed to be non-erasable and non-rewritable for the duration of its retention period; and
generating a target resident source compliance store within the target domain.
2. The method of claim 1 , further comprising:
generating a first directory and a first file in the source domain; and
in response to generating the first directory in the source domain, generating the first directory in the source compliance store.
3. The method of claim 2 , further comprising:
receiving a request to commit the first file; and
generating a canonical link to the file under the source compliance store.
4. The method of claim 3 , further comprising:
generating canonical links under the source compliance store to hard links in a set of hard links.
5. The method of claim 1 , further comprising:
determining a set of files and directories to incrementally sync based on comparing a source cluster snapshot and a target cluster snapshot;
incrementally copying files and directories form the set of files and directories from the source cluster of the computing nodes to the target cluster of the computing nodes; and
in response to copying a file among the set of files and directories that is marked as committed, adding an entry to a target cluster list structure wherein the entry includes at least a logical inode (“LIN”) of the file and a retention date.
6. The method of claim 5 , further comprising:
in response to the incremental copying completing, committing files on the target cluster of the computing nodes based on the target cluster list structure.
7. The method of claim 6 , further comprising:
in response to the committing completing, taking an ending snapshot of the target cluster of the computing nodes wherein the ending snapshot indicates a successful sync.
8. A system comprising at least one hardware processor, at least one storage device, and a memory to store instructions executed by the least one hardware processor to:
maintain on a source cluster of the computing nodes a source domain that includes a source compliance store;
maintain on a target cluster of the computing nodes a target domain that includes a target compliance store; and; and
establish a new sync between the source cluster of the computing nodes to the target cluster of the computing nodes wherein the target cluster operates as a failover target, and wherein the syncing comprises at least:
delay an application of commits by creating a new target cluster side system b-tree to track which files are to be committed during the new sync;
send pending canonical links during the new sync, wherein each pending canonical links reference a write once read many (WORM) file, wherein each canonical link is guaranteed to be non-erasable and non-rewritable for the duration of its retention period; and
generate a target resident source compliance store within the target domain.
9. The system of claim 8 , further configured to:
generate a first directory and a first file in the source domain; and
in response to generating the first directory in the source domain, generating the first directory in the source compliance store.
10. The system of claim 9 , further configured to:
receive a request to commit the first file; and
generate a canonical link to the first file under the source compliance store.
11. The system of claim 10 , further configured to:
generate canonical links under the source compliance store to hard links in the set of hard links.
12. The system of claim 8 , further configured to:
determine a set of files and directories to incrementally sync based on comparing a source cluster of the computing nodes snapshot and a target cluster of the computing nodes snapshot;
incrementally copy files and directories form the set of files and directories from the source cluster of the computing nodes to the target cluster of the computing nodes;
in response to copying a file among the set of files and directories that is marked as committed, add an entry to a target cluster list structure wherein the entry includes at least a logical inode (“LIN”) of the file and a retention date.
13. The system of claim 12 , further configured to:
in response to the incremental copying completing, commit files on the target cluster of the computing nodes based on the target cluster list structure.
14. The system of claim 13 , further configured to:
in response to the committing completing, take an ending snapshot of the target cluster of the computing nodes wherein the ending snapshot indicates a successful sync.
15. A non-transitory computer readable medium with program instructions stored thereon to perform the following acts:
maintaining on a source cluster of the computing nodes a source domain that includes a source compliance store;
maintaining on a target cluster of the computing nodes a target domain that includes a target compliance store; and
establishing a new sync between the source cluster of the computing nodes to the target cluster of the computing nodes wherein the target cluster operates as a failover target, and wherein the syncing comprises at least:
delaying an application of commits by creating a new target cluster side system b-tree to track which files are to be committed during the new sync;
sending pending canonical links during the new sync, wherein each pending canonical links reference a write once read many (WORM) file, wherein each canonical link is guaranteed to be non-erasable and non-rewritable for the duration of its retention period; and
generating a target resident source compliance store within the target domain.
16. The non-transitory computer readable medium of claim 15 , with program instructions stored thereon to further perform the following acts:
generating a first directory and a first file in the source domain; and
in response to generating the first directory in the source domain, generating the first directory in the source compliance store.
17. The non-transitory computer readable medium of claim 16 , with program instructions stored thereon to further perform the following acts:
receiving a request to commit the first file; and
generating a canonical link to the file under the source compliance store.
18. The non-transitory computer readable medium of claim 17 , with program instructions stored thereon to further perform the following acts:
generating canonical links under the source compliance store to hard links in the set of hard links.
19. The non-transitory computer readable medium of claim 15 , with program instructions stored thereon to further perform the following acts:
determining a set of files and directories to incrementally sync based on comparing a source cluster of the computing nodes snapshot and a target cluster of the computing nodes snapshot;
incrementally copying files and directories form the set of files and directories from the source cluster of the computing nodes to the target cluster of the computing nodes;
in response to copying a file among the set of files and directories that is marked as committed, adding an entry to a target cluster list structure wherein the entry includes at least a logical inode (“LIN”) of the file and a retention date.
20. The non-transitory computer readable medium of claim 19 , with program instructions stored thereon to further perform the following acts:
in response to the incremental copying completing, committing files on the target cluster of the computing nodes based on the target cluster list structure; and
in response to the committing completing, taking an ending snapshot of the target cluster of the computing nodes wherein the ending snapshot indicates a successful sync.