Managing resources in a distributed system using dynamic clusters

1. A method for performing initial placement of a data object in a distributed system that includes a plurality of hardware resources, the method comprising:

receiving a request to create an instance of a data object;

determining, in response to the request, a list of hardware resources that satisfy one or more criteria of the data object;

creating, in response to the request, a virtual cluster that includes a subset of the hardware resources included in the list of hardware resources;

selecting a hardware resource from the virtual cluster into which the data object is to be placed;

placing the data object into the hardware resource; and

releasing the virtual cluster.

2. The method of claim 1 , wherein the data object comprises a portion of a virtual computing instance.

3. The method of claim 1 , wherein the data object comprises one or more files.

4. The method of claim 3 , wherein each hardware resource comprises a different storage array.

5. The method of claim 1 , wherein the data object comprises an executable portion and each hardware resource comprises a different host system configured to execute the executable portion of the data object.

6. The method of claim 5 , wherein the executable portion of the data object is placed into a host system that is coupled to a storage array that stores the executable portion of the data object.

7. The method of claim 5 , wherein the one or more criteria of the data object includes at least one of a type requirement, a space utilization requirement, and an input/output (I/O) load requirement.

8. The method of claim 1 , wherein determining the list of hardware resources involves referencing statistical information associated with the plurality of hardware resources against one or more criteria of the data object.

9. The method of claim 8 , wherein the statistical information is managed by periodically querying a plurality of agents, wherein each agent in the plurality of agents executes on a different hardware resource included in the plurality of hardware resources.

10. The method of claim 1 , further comprising:

performing a load balancing operation across the virtual cluster that causes data objects to be transferred between the hardware resources included in the virtual cluster.

11. The method of claim 1 , wherein selecting the hardware resource comprises selecting the hardware resource at random or selecting the hardware resource based on at least a portion of the one or more criteria of the data object.

12. The method of claim 11 , wherein the one or more criteria specifies that the hardware resources do not exceed a threshold level of current utilization.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform initial placement of a data object in a distributed system that includes a plurality of hardware resources, the method comprising:

receiving a request to create an instance of a data object;

determining, in response to the request, a list of hardware resources that satisfy one or more criteria of the data object;

creating, in response to the request, a virtual cluster that includes a subset of the hardware resources included in the list of hardware resources;

selecting a hardware resource from the virtual cluster into which the data object is to be placed;

placing the data object into the hardware resource; and

releasing the virtual cluster.

14. The non-transitory computer readable medium of claim 13 , wherein the data object comprises a portion of a virtual computing instance.

15. The non-transitory computer readable medium of claim 13 , wherein the data object comprises an executable portion and each hardware resource comprises a different host system configured to execute the executable portion of the data object.

16. The non-transitory computer readable medium of claim 15 , wherein the executable portion of the data object is placed into a host system that is coupled to a storage array that stores the executable portion of the data object.

17. A system configured to perform an initial placement of a data object, comprising:

a plurality of hardware resources; and

a server machine, configured to:

receive a request to create an instance of a data object;

determine, in response to the request, a list of hardware resources that satisfy one or more criteria of the data object;

create, in response to the request, a virtual cluster that includes a subset of the hardware resources included in the list of hardware resources;

select a hardware resource from the virtual cluster into which the data object is to be placed;

place the data object into the hardware resource; and

release the virtual cluster.

18. The system of claim 17 , wherein the data object comprises a portion of a virtual computing instance.

19. The system of claim 17 , wherein the data object comprises an executable portion and each hardware resource comprises a different host system configured to execute the executable portion of the data object.

20. The system of claim 19 , wherein the executable portion of the data object is placed into a host system that is coupled to a storage array that stores the executable portion of the data object.