Backup and restore of files with multiple hard links
Methods and systems for backing up and restoring files that have multiple hard links using master file references and index node-based mappings are described. In some cases, file fetching and restoration may be performed by a storage appliance using master file references in which a master file is identified for each multi-link file that is backed-up on the storage appliance and then referenced by one or more hard links to the multi-link file. In other cases, file fetching and restoration may be performed by a storage appliance using index node-based mappings for multi-link files that provide mappings between index node identifiers (e.g., inode numbers) for the multi-link files on a primary system and hard link paths for storing the file contents of the multi-link files on a storage appliance used for backing up the primary system.
1. A method for operating a data management system, comprising:
nominating a first file of a set of files within a file system to become a first master file, the set of files comprising files to be restored to a hardware server;
identifying remaining files in the set of files within the file system, the remaining files comprising a set of non-master files including one or more non-master files that include a pointer to the first master file;
storing a respective mapping between each non-master file of the one or more non-master files and the first master file in a file metadata table;
detecting a second file of the set of non-master files that includes a pointer to a second master file, the second master file not included in the set of files to be restored;
nominating the second file to become a third master file in response to detecting that the second master file for the second file of the set of non-master files is not in the set of files to be restored;
storing one or more second mappings between a subset of the set of non-master files and the third master file in the file metadata table based at least in part on nominating the second file to become the third master file; and
restoring the set of files to the hardware server based at least in part on the one or more respective mappings and the one or more second mappings stored in the file metadata table.
2. The method of claim 1 , wherein restoring the set of files comprises:
restoring, to the hardware server, the first master file prior to detecting the second file that references to the second master file; and
restoring, to the hardware server, the one or more non-master files that reference to the first master file based at least in part on a respective link between each of the one or more non-master files and the first master file, wherein the first master file and the one or more non-master files correspond to a same device identifier and a same index node number.
3. The method of claim 1 , wherein restoring the set of files comprises:
restoring the second file based at least in part on file contents of the second master file on a storage appliance, wherein restoring the second file comprises:
transferring the second file with the file contents from the storage appliance to the hardware server.
4. The method of claim 3 , further comprising:
creating, after restoring the second file, a respective link between each non-master file of the subset of the set of non-master files and the third master file based at least in part on the third master file and each non-master file of the subset of the set of non-master files corresponding to a same device identifier and a same index node number, wherein storing the one or more second mappings comprises storing the respective link for each non-master file of the subset of the set of non-master files in a storage system.
5. The method of claim 3 , further comprising:
transmitting an instruction to link a third file of the subset of the set of non-master files to the third master file based at least in part on the third master file and each non-master file of the subset of the set of non-master files corresponding to a same device identifier and a same index node number.
6. The method of claim 3 , wherein restoring the set of files further comprises:
restoring a third file of the subset of the set of non-master files after restoring the second file and based at least in part on a second mapping between the third file and the third master file, wherein the one or more second mappings comprise the second mapping.
7. The method of claim 1 , wherein nominating the first file to become the first master file comprises:
nominating the first file to become the first master file based at least in part on the first file corresponding to a most recent creation date of a set of creation dates associated with the set of files.
8. The method of claim 1 , wherein nominating the first file to become the first master file comprises:
nominating the first file to become the first master file based at least in part on the first file corresponding to a greatest quantity of file restores of a set of quantities of file restores within a prior time period associated with the set of files.
9. The method of claim 1 , wherein the respective mapping between a non-master file of the one or more non-master files and the first master file comprises a pointer associated with an identification of a file path to the first master file.
10. The method of claim 1 , further comprising:
identifying the set of files within the file system to be restored to the hardware server; and
identifying multi-link files within the set of files based at least in part on more than one file in the file metadata table being associated with a same device identifier and a same index node number, wherein nominating the first file to become the first master file is based at least in part on identifying the multi-link files, and wherein the multi-link files comprise at least the one or more non-master files.
11. The method of claim 10 , wherein:
the multi-link files are stored within a first partition on a storage appliance; and
a second set of files are stored within a second partition on the storage appliance different than the first partition, the second set of files comprising non-linked files.
12. The method of claim 1 , wherein a master file entry for the first file in the file metadata table is empty based at least in part on nominating the first file to become the first master file.
13. A data management system, comprising:
a memory configured to store metadata associated with a set of files; and
one or more processors configured to cause the data management system to perform operations comprising:
nominating a first file of the set of files within a file system to become a first master file, the set of files comprising files to be restored to a hardware server;
identifying remaining files in the set of files within the file system, the remaining files comprising a set of non-master files including one or more non-master files that include a pointer to the first master file;
storing a respective mapping between each non-master file of the one or more non-master files and the first master file in a file metadata table;
detecting a second file of the set of non-master files that includes a pointer to a second master file, the second master file not included in the set of files to be restored;
nominating the second file to become a third master file in response to detecting that the second master file for the second file of the set of non-master files is not in the set of files to be restored;
storing one or more second mappings between a subset of the set of non-master files and the third master file in the file metadata table based at least in part on nominating the second file to become the third master file; and
restoring the set of files to the hardware server based at least in part on the one or more respective mappings and the one or more second mappings stored in the file metadata table.
14. The data management system of claim 13 , wherein, to restore the set of files to the hardware server, the one or more processors are further configured to cause the data management system to perform operations comprising:
restoring, to the hardware server, the first master file prior to detecting the second file that references to the second master file; and
restoring, to the hardware server, the one or more non-master files that reference to the first master file based at least in part on a respective link between each of the one or more non-master files and the first master file, wherein the first master file and the one or more non-master files correspond to a same device identifier and a same index node number.
15. The data management system of claim 13 , wherein, to restore the set of files to the hardware server, the one or more processors are further configured to cause the data management system to perform operations comprising:
restoring the second file based at least in part on file contents of the second master file on a storage appliance, wherein, to restore the second file, the one or more processors are configured to cause the data management system to perform operations comprising:
transferring the second file with the file contents from the storage appliance to the hardware server.
16. The data management system of claim 15 , wherein the one or more processors are further configured to cause the data management system to perform operations comprising:
creating, after restoring the second file, a respective link between each non-master file of the subset of the set of non-master files and the third master file based at least in part on the third master file and each non-master file of the subset of the set of non-master files corresponding to a same device identifier and a same index node number, wherein storing the one or more second mappings comprises storing the respective link for each non-master file of the subset of the set of non-master files in a storage system.
17. A non-transitory machine-readable medium including instructions which, when read by a system cause the system to perform operations comprising:
nominating a first file of a set of files within a file system to become a first master file, the set of files comprising files to be restored to a hardware server;
identifying remaining files in the set of files within the file system, the remaining files comprising a set of non-master files including one or more non-master files that include a pointer to the first master file;
storing a respective mapping between each non-master file of the one or more non-master files and the first master file in a file metadata table;
detecting a second file of the set of non-master files that includes a pointer to a second master file, the second master file not included in the set of files to be restored;
nominating the second file to become a third master file in response to detecting that the second master file for the second file of the set of non-master files is not in the set of files to be restored;
storing one or more second mappings between a subset of the set of non-master files and the third master file in the file metadata table based at least in part on nominating the second file to become the third master file; and
restoring the set of files to the hardware server based at least in part on the one or more respective mappings and the one or more second mappings stored in the file metadata table.
18. The non-transitory machine-readable medium of claim 17 , wherein, to restore the set of files to the hardware server, the instructions further cause the system to perform operations, comprising:
restoring, to the hardware server, the first master file prior to detecting the second file that references to the second master file; and
restoring, to the hardware server, the one or more non-master files that reference to the first master file based at least in part on a respective link between each of the one or more non-master files and the first master file, wherein the first master file and the one or more non-master files correspond to a same device identifier and a same index node number.
19. The non-transitory machine-readable medium of claim 17 , wherein, to restore the set of files to the hardware server, the instructions further cause the system to perform operations, comprising:
restoring the second file based at least in part on file contents of the second master file on a storage appliance, wherein, to restore the second file, the instructions cause the system to perform operations comprising:
transferring the second file with the file contents from the storage appliance to the hardware server.
20. The non-transitory machine-readable medium of claim 19 , wherein the instructions further cause the system to perform operations, comprising:
creating, after restoring the second file, a respective link between each non-master file of the subset of the set of non-master files and the third master file based at least in part on the third master file and each non-master file of the subset of the set of non-master files corresponding to a same device identifier and a same index node number, wherein storing the one or more second mappings comprises storing the respective link for each non-master file of the subset of the set of non-master files in a storage system.