Agentless virtual disk metadata indexing
A method and system for implementing agentless virtual disk metadata indexing. Specifically, the disclosed method and system entails the parsing of data within a virtual machine and/or virtual disk snapshot to extract filesystem metadata information during data backup operations. Extraction of the filesystem metadata information is performed based on an understanding of a filesystem layout for a format in which the snapshot is defined, rather than based on the mounting and traversing of the snapshot. Furthermore, the extracted filesystem metadata information is encoded in a schema that may facilitate indexing of the information for use in a metadata searching and/or retrieval service.
1. A method for obtaining virtual disk metadata (VDM), comprising:
issuing a snapshot request comprising a virtual machine (VM) identifier;
receiving, in response to the snapshot request, a virtual disk image (VDI) for a virtual disk associated with the VM identifier;
parsing the VDI based on a filesystem layout for a format in which the VDI is received, wherein the filesystem layout is for a filesystem;
obtaining, from parsing the VDI, virtual disk data (VDD) and the VDM for the virtual disk;
generating an intermediate data object (IDO) using the VDM;
processing the IDO by building a full path for each directory and for content within each directory for the filesystem to obtain an IDO index, wherein the IDO index comprises a data structure in which the full path is stored; and
employing the IDO index to implement a VDM searching service.
2. The method of claim 1 , wherein the VM identifier is associated with a VM implemented using at least the virtual disk.
3. The method of claim 1 , wherein the VDI comprises the filesystem storing the VDD and the VDM, wherein the VDD comprises raw data in the filesystem, wherein the VDM comprises metadata describing the filesystem.
4. The method of claim 1 , further comprising:
prior to processing of the IDO:
generating a virtual disk backup (VDB) using the VDD and the IDO; and
storing the VDB in backup storage wherefrom the IDO of the VDB is retrieved for processing.
5. The method of claim 1 , wherein generating the IDO comprises encoding the VDM according to a schema, wherein the schema is based on a structured query language (SQL)ite relational database file format.
6. A system, comprising:
a virtual machine (VM) manager and backup storage;
a backup proxy node (BPN) operatively connected to the VM manager and the backup storage, and programmed to:
issue, to the VM manager, a snapshot request comprising a VM identifier;
receive, from the VM manager and in response to the snapshot request, a virtual disk image (VDI) relating to a virtual disk associated with the VM identifier;
parse the VDI based on a filesystem layout for a format in which the VDI is received, wherein the filesystem layout is for a filesystem;
obtain, from parsing the VDI, virtual disk data (VDD) and virtual disk metadata (VDM) for the virtual disk;
generate an intermediate data object (IDO) using the VDM; and
store, in the backup storage, a virtual disk backup (VDB) comprising the VDD and the IDO, and
an application programming interface (API) operatively connected to the backup storage, and programmed to:
issue, to the backup storage, a restore request comprising the VM identifier;
receive the IDO, extracted from the VDB, from the backup storage and in response to the restore request;
process the IDO by building a full path for each directory and for content within each directory for the filesystem to obtain an IDO index, wherein the IDO index comprises a data structure in which the full path is stored; and
employ the IDO index to implement a VDM searching service.
7. The system of claim 6 , further comprising:
a cloud search service (CSS) comprising the API;
and
a VM host comprising the VM manager and at least one VM.
8. A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to:
issue a snapshot request comprising a virtual machine (VM) identifier;
receive, in response to the snapshot request, a virtual disk image (VDI) for a virtual disk associated with the VM identifier;
parse the VDI based on a filesystem layout for a format in which the VDI is received, wherein the filesystem layout is for a filesystem; and
obtain, from parsing the VDI, virtual disk data (VDD) and the VDM for the virtual disk;
generate an intermediate data object (IDO) using the VDM;
process the IDO by building a full path for each directory and for content within each directory for the filesystem to obtain an IDO index, wherein the IDO index comprises a data structure in which the full path is stored; and
employ the IDO index to implement a VDM searching service.
9. The non-transitory CRM of claim 8 , wherein the VM identifier is associated with a VM implemented using at least the virtual disk.
10. The non-transitory CRM of claim 8 , wherein the VDI comprises the filesystem storing the VDD and the VDM, wherein the VDD comprises raw data in the filesystem, wherein the VDM comprises metadata describing the filesystem.
11. The non-transitory CRM of claim 8 , comprising computer readable program code, which when executed by the computer processor, further enables the computer processor to:
prior to processing of the IDO:
generate a virtual disk backup (VDB) using the VDD and the IDO; and
store the VDB in backup storage wherefrom the IDO of the VDB is retrieved for processing.
12. The non-transitory CRM of claim 8 , wherein generating the IDO comprises encoding the VDM according to a schema, wherein the schema is based on a structured query language (SQL)ite relational database file format.