Efficient mass virtual machine (VM) cloning
In one set of embodiments, during an initial pre-copying phase, copies of the virtual disks of a VM or VM template can be distributed across one or more datastores, prior to the start of any cloning. This effectively seeds each datastore with a disk pool comprising a certain number of copies of the VM/VM template's virtual disks. Then, at the time of receiving a user request to create k clones from the VM/VM template, up to k copies of the VM/VM template's virtual disks that already reside in the disk pools of one or more target datastores can be moved from those disk pools to the clones' home folders on the target datastores, thereby reducing or eliminating the need to create brand new copies of the virtual disks on demand.
1. On a network environment comprising a management server having a disk pool manager, and a plurality of cluster host environments wherein each cluster host environment comprises a disk pool agent, a method comprising:
receiving, by a management server, a registration request for registering a virtual machine (VM) or VM template as a source for future VM cloning operations, wherein the VM or VM template is associated with a virtual disk, and wherein the registration request includes a content distribution policy specifying one or more datastores in a cluster and a desired number of copies of the virtual disk to be created in a disk pool of each of the one or more datastores;
identifying, by the disk pool manager, a subset of the plurality of host systems in the cluster with access to the one or more datastores; and
transferring, by the disk pool manager, a seed copy of the virtual disk to each of the subset of the plurality of host systems, and
operating the seed copy of the virtual disk by a hardware processor of each respective host system of the subset of the plurality of host systems, wherein operating the seed copy for each subset of host systems includes storing the seed copy in a disk pool of a datastore in the one or more datastores and creates, using the seed copy, a plurality of additional local copies of the virtual disk in the disk pool for the respective host system;
for each disk pool agent:
iteratively comparing, by the respective disk pool agent, a number of available copies in a disk pool, for the respective cluster host environment, to a minimum threshold number specified in a content distribution policy template;
wherein if the number of available copies in a disk pool for the respective cluster host environment is less than a minimum threshold number:
notifying the management server of the identified respective cluster host environments need for additional clones of the seed copy; and
coordinating, by the disk pool manager, for sending of additional clones to the identified cluster host environment from other host environments associated with the one or more data stores.
2. The method of claim 1 further comprising, subsequently to the transferring:
receiving a clone request for creating k clones of the VM or VM template;
identifying a target datastore in the one or more datastores holding at least k copies of the virtual disk in its disk pool;
identifying a host system with access to the target datastore; and
sending a clone command to the host system with instructions for moving the k copies from the disk pool of the target datastore to VM home folders of the k clones.
3. The method of claim 1 wherein the seed copy is transferred to each of the one or more host systems using a network multicast protocol.
4. The method of claim 1 wherein the transferring of the seed copy to each of the one or more host systems comprises reading the virtual disk exactly once.
5. The method of claim 1 wherein said each host system creates the one or more additional local copies by reading the seed copy exactly once.
6. A non-transitory computer readable storage medium comprising nonvolatile memory having stored thereon program code executable by a computer system, the program code embodying a method comprising:
receiving, by a management server, a registration request for registering a virtual machine (VM) or VM template as a source for future VM cloning operations, wherein the VM or VM template is associated with a virtual disk, and wherein the registration request includes a content distribution policy specifying one or more datastores in a cluster and a desired number of copies of the virtual disk to be created in a disk pool of each of the one or more datastores;
identifying, by the disk pool manager, a subset of the plurality of host systems in the cluster with access to the one or more datastores; and
transferring, by the disk pool manager, a seed copy of the virtual disk to each of the subset of the plurality of host systems, and
operating the seed copy of the virtual disk by a hardware processor of each respective host system of the subset of the plurality of host systems, wherein operating the seed copy for each subset of host systems includes storing the seed copy in a disk pool of a datastore in the one or more datastores and creates, using the seed copy, a plurality of additional local copies of the virtual disk in the disk pool for the respective host system;
for each disk pool agent:
iteratively comparing, by the respective disk pool agent, a number of available copies in a disk pool, for the respective cluster host environment, to a minimum threshold number specified in a content distribution policy template;
wherein if the number of available copies in a disk pool for the respective cluster host environment is less than a minimum threshold number:
notifying the management server of the identified respective cluster host environments need for additional clones of the seed copy; and
coordinating, by the disk pool manager, for sending of additional clones to the identified cluster host environment from other host environments associated with the one or more data stores.
7. The non-transitory computer readable storage medium of claim 6 wherein the method further comprises, subsequently to the transferring:
receiving a clone request for creating k clones of the VM or VM template;
identifying a target datastore in the one or more datastores holding at least k copies of the virtual disk in its disk pool;
identifying a host system with access to the target datastore; and
sending a clone command to the host system with instructions for moving the k copies from the disk pool of the target datastore to VM home folders of the k clones.
8. The non-transitory computer readable storage medium of claim 6 wherein the seed copy is transferred to each of the one or more host systems using a network multicast protocol.
9. The non-transitory computer readable storage medium of claim 6 wherein the transferring of the seed copy to each of the one or more host systems comprises reading the virtual disk exactly once.
10. The non-transitory computer readable storage medium of claim 6 wherein said each host system creates the one or more additional local copies by reading the seed copy exactly once.
11. A computer system comprising:
a processor; and
a non-transitory computer readable medium having stored thereon program code that causes the processor to:
receive a registration request for registering a virtual machine (VM) or VM template as a source for future VM cloning operations, wherein the VM or VM template is associated with a virtual disk, and wherein the registration request includes a content distribution policy specifying one or more datastores in a cluster and a desired number of copies of the virtual disk to be created in a disk pool of each of the one or more datastores;
identify, by a disk pool manager, a subset of the plurality of systems in the cluster with access to the one or more datastores; and
transfer a seed copy of the virtual disk to each of the subset of the plurality of host systems, and
operating the seed copy of the virtual disk by a hardware processor of each respective host system of the subset of the plurality of host systems, wherein operating the seed copy for each subset of host systems includes storing the seed copy in a disk pool of a datastore in the one or more datastores and creates, using the seed copy, a plurality of additional local copies of the virtual disk in the disk pool for the respective host system;
for each disk pool agent:
iteratively comparing, by the respective disk pool agent, a number of available copies in a disk pool, for the respective cluster host environment, to a minimum threshold number specified in a content distribution policy template;
wherein if the number of available copies in a disk pool for the respective cluster host environment is less than a minimum threshold number:
notifying the management server of the identified respective cluster host environments need for additional clones of the seed copy; and
coordinating, by the disk pool manager, for sending of additional clones to the identified cluster host environment from other host environments associated with the one or more data stores.
12. The computer system of claim 11 wherein the program code further causes the processor to, subsequently to the transferring:
receive a clone request for creating k clones of the VM or VM template;
identify a target datastore in the one or more datastores holding at least k copies of the virtual disk in its disk pool;
identify a host system with access to the target datastore; and
send a clone command to the host system with instructions for moving the k copies from the disk pool of the target datastore to VM home folders of the k clones.
13. The computer system of claim 11 wherein the seed copy is transferred to each of the one or more host systems using a network multicast protocol.
14. The computer system of claim 11 wherein the transferring of the seed copy to each of the one or more host systems comprises reading the virtual disk exactly once.
15. The computer system of claim 11 wherein said each host system creates the one or more additional local copies by reading the seed copy exactly once.