Optimized copy of virtual machine storage files
A facility is described for optimizing the copying of virtual machine storage files. In various embodiments, the facility refrains from copying portions of the virtual machine storage file that do not contain any meaningful data for the purposes of the guest file system within the virtual machine storage file. In some embodiments, the facility refrains from copying portions of the file that are in use by the guest file system, but are of no practical value.
1. A method performed by a host computer system for performing an optimized copy of virtual machine storage files, the method comprising:
under control of a physical host computer system configured to execute virtual machines associated with virtual machine storage files, the physical host computer system having executing thereon a physical host operating system that provides one or more application programming interfaces (APIs):
mounting, as a volume, on the physical host computer system, a virtual machine storage file associated with one of the virtual machines;
determining a handle of the mounted volume via the one or more APIs of the physical host operating system;
wherein the virtual machine storage file comprises guest system metadata and a guest file system;
wherein the guest file system comprises a plurality of logical clusters having a file system cluster size expressible in bytes;
wherein the guest system metadata comprises a cluster bitmap, the cluster bitmap comprising information on whether each logical cluster of the guest file system is in use by the guest file system;
identifying the guest file system included in the virtual machine storage file via the handle and the one or more APIs of the physical host operating system;
obtaining, via the one or more APIs of the physical host operating system, the file system cluster size of the guest file system and a number of logical clusters in the plurality of logical clusters of the guest file system;
identifying, via the one or more APIs of the physical host operating system, a location within the virtual machine storage file of the first logical cluster of the guest file system;
copying, to a destination file, the virtual machine storage file from a beginning of the virtual machine storage file to the identified location of the first logical cluster of the guest file system;
accessing the cluster bitmap for information on whether each logical cluster in the guest file system is in use by the guest file system;
for each logical cluster of the plurality of logical clusters, determining, from the cluster bitmap, whether the logical cluster is in use by the guest file system;
for each logical cluster of the plurality of logical clusters, responsive to a determination that the logical cluster is not in use by the guest file system, advancing file pointer of the destination file by the file system cluster size without copying the logical cluster to the destination file; and
for each logical cluster of the plurality of logical clusters, responsive to a determination that the logical cluster is in use by the guest file system, copying the logical cluster to the destination file.
2. The method of claim 1 further comprising:
determining from the accessed cluster bitmap whether additional clusters remain to be processed; and
if additional clusters remain to be processed, further processing the cluster bitmap.
3. The method of claim 1 further comprising:
identifying a location within the virtual machine storage file of the last cluster of the guest file system; and
copying, to the destination file, the virtual machine storage file from the identified location of the last cluster of the guest file system to an end of the virtual machine storage file.
4. The method of claim 3 further comprising setting file meta data.
5. The method of claim 1 further comprising avoiding copying blocks occupied by a hibernation file or a paging file associated with the guest file system.
6. The method of claim 1 wherein the virtual machine storage file being copied comprises a guest operating system image associated with the guest file system.
7. The method of claim 1 wherein the virtual machine storage file being copied comprises the guest file system, but does not comprise a guest operating system image.
8. The method of claim 1 wherein the virtual machine storage file being copied comprises a single volume embedded within the virtual machine storage file.
9. The method of claim 1 wherein the virtual machine storage file being copied comprises multiple volumes embedded within the virtual machine storage file.
10. The method of claim 1 wherein the virtual machine storage file comprises a Microsoft specified VHD file format.
11. The method of claim 1 wherein the virtual machine storage file comprises a VMware specified VMDK file format.
12. The method of claim 1 wherein the virtual machine storage file is copied to multiple destination files.
13. The method of claim 1 further comprising:
determining, from the accessed cluster bitmap, clusters in the guest file system that are free; and
filling with zeroes portions of the destination file associated with the clusters determined to be free.
14. The method of claim 1 wherein accessing the virtual machine storage file comprises accessing the virtual machine storage file as a read-only volume attached to a host operating system of the host computer system.
15. A system for copying virtual machine storage files, the system comprising a physical host computer system configured to execute a virtual machine, the virtual machine associated with a guest operating system and a guest file system, the physical host computer system having executing thereon a physical host operating system that provides one or more application programming interfaces (APIs), the physical host computing system configured to implement a method comprising:
mounting, as a volume, on the physical host computer system, a virtual machine storage file associated with one of the virtual machines;
determining a handle of the mounted volume via the one or more APIs of the physical host operating system;
wherein the virtual machine storage file comprises guest system metadata and a guest file system;
wherein the guest file system comprises a plurality of logical clusters having a file system cluster size expressible in bytes;
wherein the guest system metadata comprises a cluster bitmap, the cluster bitmap comprising information on whether each logical cluster of the guest file system is in use by the guest file system;
identifying the guest file system included in the virtual machine storage file via the handle and the one or more APIs of the physical host operating system;
obtaining, via the one or more APIs of the physical host operating system, the file system cluster size of the guest file system and a number of logical clusters in the plurality of logical clusters of the guest file system;
identifying, via the one or more APIs of the physical host operating system, a location within the virtual machine storage file of the first logical cluster of the guest file system;
copying, to a destination file, the virtual machine storage file from a beginning of the virtual machine storage file to the identified location of the first logical cluster of the guest file system;
accessing the cluster bitmap for information on whether each logical cluster in the guest file system is in use by the guest file system;
for each logical cluster of the plurality of logical clusters, determining, from the cluster bitmap, whether the logical cluster is in use by the guest file system;
for each logical cluster of the plurality of logical clusters, responsive to a determination that the logical cluster is not in use by the guest file system, advancing file pointer of the destination file by the file system cluster size without copying the logical cluster to the destination file; and
for each logical cluster of the plurality of logical clusters, responsive to a determination that the logical cluster is in use by the guest file system, copying the logical cluster to the destination file.
16. A computer-program product comprising a non-transitory computer-usable medium having computer-readable program code embodied therein, the computer-readable program code adapted to be executed to implement a method, the method being performed by an agent installed in a computing environment on a computer system, the method comprising:
mounting, as a volume, on a physical host computer system, a virtual machine storage file associated with a virtual machine;
determining a handle of the mounted volume via one or more APIs provided by a physical host operating system executing on the physical host computer system;
wherein the virtual machine storage file comprises guest system metadata and a guest file system;
wherein the guest file system comprises a plurality of logical clusters having a file system cluster size expressible in bytes;
wherein the guest system metadata comprises a cluster bitmap, the cluster bitmap comprising information on whether each logical cluster of the guest file system is in use by the guest file system;
identifying the guest file system included in the virtual machine storage file via the handle and the one or more APIs of the physical host operating system;
obtaining, via the one or more APIs of the physical host operating system, the file system cluster size of the guest file system and a number of logical clusters in the plurality of logical clusters of the guest file system;
identifying, via the one or more APIs of the physical host operating system, a location within the virtual machine storage file of the first logical cluster of the guest file system;
copying, to a destination file, the virtual machine storage file from a beginning of the virtual machine storage file to the identified location of the first logical cluster of the guest file system;
accessing the cluster bitmap for information on whether each logical cluster in the guest file system is in use by the guest file system;
for each logical cluster of the plurality of logical clusters, determining, from the cluster bitmap, whether the logical cluster is in use by the guest file system;
for each logical cluster of the plurality of logical clusters, responsive to a determination that the logical cluster is not in use by the guest file system, advancing file pointer of the destination file by the file system cluster size without copying the logical cluster to the destination file; and
for each logical cluster of the plurality of logical clusters, responsive to a determination that the logical cluster is in use by the guest file system, copying the logical cluster to the destination file.