Cataloging system for image-based backup
Examples of systems and methods for cataloging image-based backups are disclosed. In certain implementations, these systems and methods can obtain information about file system structure from a volume undergoing image-based backup. The systems and methods can generate a catalog of the file system structure that can be stored with the backup image. The systems and methods can generate a searchable index of the catalog so that users can search for files within the backup image without being required to manually mount the backup image containing the searched-for file. As a result, in certain embodiments, users advantageously can more quickly locate and restore files from image-based backups.
1. A system for backing up a virtual machine disk file in a virtual computing environment, the system comprising:
a computer system comprising computer hardware, the computer system comprising a data store, the computer system programmed to implement a cataloging module, a backup module, and a restore module, wherein:
the cataloging module is configured to, for each virtual machine disk file of a plurality of virtual machine disk files:
access the virtual machine disk file comprising a file system of a guest operating system of a virtual machine to obtain metadata about the file system, the metadata comprising information about a logical structure of the file system;
analyze the metadata to determine a logical hierarchy of a set of files in the file system;
analyze the metadata to determine one or more attributes of the set of files in the file system;
generate a catalog comprising information about the logical hierarchy and the one or more attributes of the set of files in the file system, the catalog comprising a computer-enabled searchable index of at least one of a plurality of image-based backups, wherein the computer-enabled searchable index performs automated file searches within the plurality of image-based backups without manually mounting a backup image containing a searched-for file;
wherein each image-based backup of the plurality of image-based backups is associated with a unique catalog; and
store the catalog outside the virtual machine;
the backup module is configured to, for each virtual machine disk file of the plurality of virtual machine disk files:
backup the virtual machine disk file to a backup image stored on the data store; and
the restore module is configured to:
enable a user to search the catalogs for a desired file from a virtual machine image-based backup from outside the virtual machines and without specifying which virtual machine image includes the desired file; and
wherein the computer system is programmed to concurrently and asynchronously execute the cataloging module in a first thread and the backup module in a second thread different from the first thread, wherein the cataloging module is configured to determine the logical hierarchy of the at least some files in the file system in a first pass through the metadata and to determine the one or more attributes of the at least some files in the file system in a second pass through the metadata; and
wherein the first thread and the second thread are executed by a backup server outside the virtual machine associated with the virtual machine disk file.
2. The system of claim 1 , wherein the set of files includes at least one file and at least one directory of the file system.
3. The system of claim 1 , wherein a first processor is programmed to execute the cataloging module and the backup module, and a second processor is programmed to execute the virtual machine, the first processor different from the second processor.
4. The system of claim 1 , wherein the cataloging module is configured to store the catalog in the data store.
5. The system of claim 1 , wherein the computer system is programmed to execute the cataloging module and the backup module outside the virtual machine.
6. The system of claim 1 , wherein the metadata comprises a master file table (MFT).
7. The system of claim 1 , wherein the virtual machine disk file comprises a snapshot of a base virtual machine disk file, the snapshot comprising a point-in-time copy of the base file.
8. The system of claim 7 , wherein the backup module is further configured to establish a savepoint, the savepoint comprising the backup image and the catalog.
9. The system of claim 1 , wherein the set of files includes all the files and directories of the file system.
10. The system of claim 1 , wherein the catalog or the computer-enabled searchable index includes path information for the set of files in the file system.
11. The system of claim 1 , wherein the restore module is configured to:
receive a search inquiry for a file that may be included in a backup image, the backup image associated with a catalog;
generate a searchable index from the catalog associated with the backup image;
search the searchable index of the catalog associated with the backup image to determine matches to the search inquiry; and
output information relating to the matches to the search inquiry.
12. The system of claim 11 , wherein the restore module is further configured to:
receive a selection of a match to the search inquiry;
mount the backup image associated with the selection of the match to the search inquiry; and
restore the selection from the mounted backup image to a target.
13. The system of claim 11 , wherein the computer system is programmed to implement a database module, and to generate the computer-enabled searchable index from the catalog associated with the backup image:
the restore module is configured to import the catalog into the database module; and
the database module is configured to generate the computer-enabled searchable index.
14. A method of backing up a virtual machine disk file in a virtual computing environment, the method comprising:
under control of a computer system comprising one or more physical computing devices, for each virtual machine disk file of a plurality of virtual machine disk files:
accessing metadata of the virtual machine disk file, the virtual machine disk file comprising a file system of a guest operating system of a virtual machine executing on a host server, the metadata comprising information about a logical structure of the file system;
analyzing the metadata to determine a logical hierarchy of a set of files in the file system and one or more attributes of the set of files in the file system;
filtering the metadata based on at least one filter criterion to determine the set of files in the file system;
generating a catalog comprising information about the logical hierarchy and the one or more attributes of the set of files in the file system, the catalog comprising a computer-enabled searchable index of at least one of a plurality of image-based backups, wherein the computer-enabled searchable index performs automated file searches within the plurality of image-based backups without manually mounting a backup image containing a searched-for file;
wherein each image-based backup of the plurality of image-based backups is associated with a unique catalog;
storing the catalog in a persistent storage outside the virtual machine;
backing up the virtual machine disk file to a backup image;
the computer system enabling a user to search the catalogs for a desired file from a virtual machine image-based backup from outside the virtual machines and without specifying which virtual machine image includes the desired file;
wherein the computer system is programmed to concurrently and asynchronously execute a first thread and a second thread;
wherein the first thread comprises a cataloging thread that is configured to determine the logical hierarchy of the at least some files in the file system in a first pass through the metadata and to determine the one or more attributes of the at least some files in the file system in a second pass through the metadata;
wherein the second thread comprises the backing up; and
wherein the first thread and the second thread are executed by a backup server outside the virtual machine associated with the virtual machine disk file.
15. The method of claim 14 , wherein backing up the virtual machine disk file comprises storing the backup image in the persistent storage.
16. The method of claim 14 , wherein analyzing the metadata comprises reading the metadata a plurality of times.
17. The method of claim 14 , further comprising:
wherein the virtual machine disk file comprises a snapshot of a base virtual machine disk file, the snapshot comprising a point-in-time copy of the base virtual machine disk file; and
establishing a savepoint, the savepoint comprising the backup image and the catalog.
18. The method of claim 14 , further comprising:
searching the computer-enabled searchable index to find a match to a user inquiry for a file that may be stored in the backup image; and
if a match is found, restoring the file from the backup image to a target destination.
19. A computer-readable storage medium comprising computer-executable instructions configured to implement a method of backing up a virtual machine image, the method comprising:
for each virtual machine image of a plurality of virtual machine images:
accessing metadata of the virtual machine image, the virtual machine image comprising a file system of a guest operating system of a virtual machine, the metadata comprising information about a logical structure of the file system;
analyzing the metadata to determine a logical hierarchy of a set of files in the file system and one or more attributes of the set of files in the file system;
filtering the metadata based on at least one filter criterion to determine the set of files in the file system;
generating a catalog comprising information about the logical hierarchy and the one or more attributes of the set of files in the file system, the catalog comprising a computer-enabled searchable index of at least one of a plurality of image-based backups, wherein the computer-enabled searchable index performs automated file searches within the plurality of image-based backups without manually mounting a backup image containing a searched-for file;
wherein each image-based backup of the plurality of image-based backups is associated with a unique catalog;
storing the catalog in a persistent storage;
backing up the virtual machine image to a backup image;
enabling a user to search the catalogs for a desired file from a virtual machine image-based backup from outside the virtual machines and without specifying which virtual machine image includes the desired file;
wherein the computer-executable instructions launch a first thread and a second thread that execute concurrently and asynchronously;
wherein the first thread comprises a cataloging thread that is configured to determine the logical hierarchy of the at least some files in the file system in a first pass through the metadata and to determine the one or more attributes of the at least some files in the file system in a second pass through the metadata;
wherein the second thread comprises the backing up; and
wherein the first thread and the second thread are executed by a backup server outside the virtual machine associated with a virtual machine disk file.