Methods and subsystems that efficiently distribute VM images in distributed computing systems
The current document is directed to methods and systems that efficiently distribute virtual-machine images (“VM images”) among servers within large, distributed-computer-system-implemented IAAS platforms to facilitate temporally and computationally efficient instantiation of virtual machines within the servers. In implementations discussed below, VM images are stored in a distributed fashion throughout one or more distributed computing systems, using several different VM-image-distribution models, in order to balance computational-resource usage, temporal constraints, and other factors and considerations related to VM-image distribution and VM instantiation.
1. A distributed VM-image-distribution subsystem of a management system within a distributed computer system having multiple servers, multiple data-storage devices, and one or more internal networks, the distributed VM-image-distribution subsystem implemented as stored computer instructions that, when executed on one or more processors of one or more computer systems, control the computer system to:
organize data stores within the distributed computer system that store VM images into groups, each group including an image data store and multiple associated local data stores;
classify each VM image as one of an EAGER image and an ON_DEMAND image;
when an EAGER image is created and stored in a first image data store, propagate the VM image to the remaining image data stores within the distributed computer system and propagate the VM image from each image data store to the local data stores associated with the image data store;
when an ON_DEMAND image is created and stored in a first image data store, propagate the VM image to the remaining image data stores within the distributed computer system; and
when the management system processes a request to instantiate an ON_DEMAND image on a server, transfers the ON_DEMAND image from an image data store to a local data store accessible to the server.
2. The distributed VM-image-distribution subsystem of claim 1 further comprising:
local distributed-VM-image-distribution instances, each executing within a management server and each associated with an image data store and multiple associated local data stores.
3. The distributed VM-image-distribution subsystem of claim 2 wherein each local distributed-VM-image-distribution instance includes a background subsystem responsible for initiating batches of VM-image copy operations through a management network and an orchestrator subsystem that manages VM-image propagation through the distributed computer system.
4. The distributed VM-image-distribution subsystem of claim 2 wherein, when an ON_DEMAND image is created and stored in a first image data store and when the ON_DEMAND image has been propagated to the remaining image data stores within the distributed computer system, the distributed VM-image-distribution subsystem generates a signal to the management system, following reception of which the management system indicates successful completion of creation of the ON_DEMAND image and availability of the ON_DEMAND image for instantiation.
5. The distributed VM-image-distribution subsystem of claim 2 wherein, when, following creation of an EAGER image and storing of the EAGER image in a first image data store, the EAGER image is successfully propagated to the remaining image data stores within the distributed computer system and propagated to a first batch of local data stores associated with each image data store, the distributed VM-image-distribution subsystem generates a signal to the management system, following reception of which the management system indicates successful completion of creation of the EAGER image and availability of the EAGER image for instantiation.
6. The distributed VM-image-distribution subsystem of claim 2 wherein EAGER images are restricted to sizes below a threshold size.
7. The distributed VM-image-distribution subsystem of claim 2 wherein sizes of ON_DEMAND images are not restricted.
8. The distributed VM-image-distribution subsystem of claim 2 wherein the EAGER and ON_DEMAND classifications of VM images are one of:
fixed when VM images are created; and
initially assigned when VM images are created and later modified, by the management system, when the instantiation pattern of a VM image is incompatible with the classification assigned to the VM image.
9. A method carried out in a distributed VM-image-distribution subsystem of a management system within a distributed computer system having multiple servers, multiple data-storage devices, and one or more internal networks, the method comprising:
organizing data stores within the distributed computer system that store VM images into groups, each group including an image data store and multiple associated local data stores;
classifying each VM image as one of an EAGER image and an ON_DEMAND image;
when an EAGER image is created and stored in a first image data store, propagating the VM image to the remaining image data stores within the distributed computer system and propagating the VM image from each image data store to the local data stores associated with the image data store;
when an ON_DEMAND image is created and stored in a first image data store, propagating the VM image to the remaining image data stores within the distributed computer system; and
when the management system processes a request to instantiate an ON_DEMAND image on a server, transferring the ON_DEMAND image from an image data store to a local data store accessible to the server.
10. The method of claim 9 wherein the distributed-VM-image-distribution subsystem comprises:
local distributed-VM-image-distribution instances, each executing within a management server and each associated with an image data store and multiple associated local data stores.
11. The method of claim 10 wherein each local distributed-VM-image-distribution instance includes a background subsystem responsible for initiating batches of VM-image copy operations through a management network and an orchestrator subsystem that manages VM-image propagation through the distributed computer system.
12. The method of claim 11 wherein, when an ON_DEMAND image is created and stored in a first image data store and when the ON_DEMAND image has been propagated to the remaining image data stores within the distributed computer system, generating a signal to the management system, following reception of which the management system indicates successful completion of creation of the ON_DEMAND image and availability of the ON_DEMAND image for instantiation.
13. The method of claim 10 wherein, when, following creation of an EAGER image and storing of the EAGER image in a first image data store, the EAGER image is successfully propagated to the remaining image data stores within the distributed computer system and propagated to a first batch of local data stores associated with each image data store, generating a signal to the management system, following reception of which the management system indicates successful completion of creation of the EAGER image and availability of the EAGER image for instantiation.
14. The method of claim 10 wherein EAGER images are restricted to sizes below a threshold size.
15. The method of claim 10 wherein sizes of ON_DEMAND images are not restricted.
16. The method of claim 10 wherein the EAGER and ON_DEMAND classifications of VM images are one of:
fixed when VM images are created; and
initially assigned when VM images are created and later modified, by the management system, when the instantiation pattern of a VM image is incompatible with the classification assigned to the VM image.
17. Computer instructions stored on a physical data-storage device that, when executed by one or more processors in a distributed computer system having multiple servers, multiple data-storage devices, and one or more internal networks, control a distributed VM-image-distribution subsystem of a management system within the distributed computer system to:
organize data stores within the distributed computer system that store VM images into groups, each group including an image data store and multiple associated local data stores;
classify each VM image as one of an EAGER image and an ON_DEMAND image;
when an EAGER image is created and stored in a first image data store, propagate the VM image to the remaining image data stores within the distributed computer system and propagate the VM image from each image data store to the local data stores associated with the image data store;
when an ON_DEMAND image is created and stored in a first image data store, propagate the VM image to the remaining image data stores within the distributed computer system; and
when the management system processes a request to instantiate an ON_DEMAND image on a server, transfers the ON_DEMAND image from an image data store to a local data store accessible to the server.
18. The stored computer instructions of claim 17 wherein the distributed-VM-image-distribution subsystem comprises local distributed-VM-image-distribution instances, each executing within a management server and each associated with an image data store and multiple associated local data stores; and
wherein each local distributed-VM-image-distribution instance includes a background subsystem responsible for initiating batches of VM-image copy operations through a management network and an orchestrator subsystem that manages VM-image propagation through the distributed computer system.
19. The stored computer instructions of claim 17 wherein, when an ON_DEMAND image is created and stored in a first image data store and when the ON_DEMAND image has been propagated to the remaining image data stores within the distributed computer system, generating a signal to the management system, following reception of which the management system indicates successful completion of creation of the ON_DEMAND image and availability of the ON_DEMAND image for instantiation.
20. The stored computer instructions of claim 17 wherein, when, following creation of an EAGER image and storing of the EAGER image in a first image data store, the EAGER image is successfully propagated to the remaining image data stores within the distributed computer system and propagated to a first batch of local data stores associated with each image data store, generating a signal to the management system, following reception of which the management system indicates successful completion of creation of the EAGER image and availability of the EAGER image for instantiation.