Composable infrastructure provisioning and balancing

1. A method comprising:

receiving a request to provision one or more system components for execution of a job;

calculating aggregate latencies of a plurality of computing units based on startup latencies for jobs executing on the plurality of computing units;

selecting a computing unit from among the plurality of computing units based on the aggregate latencies; and

provisioning system components from the computing unit.

2. The method of claim 1 , wherein calculating the aggregate latencies of the plurality of computing units further comprises:

receiving load metrics specifying at least a time to job availability for each job executing on the computing unit.

3. The method of claim 2 , wherein the startup latencies include the time to job availability.

4. The method of claim 1 , wherein the startup latencies further include a time to network connection.

5. The method of claim 4 , wherein the time to network connection is determined by:

transmitting a network message into a network port of the job;

listening to the network port of the job until a reply is received; and

calculating the time to network connection as the time from when the network message is transmitted until the reply is received.

6. The method of claim 1 , wherein calculating the aggregate latencies further comprises:

weighting the startup latencies for jobs executing on the plurality of computing units with a weight that varies according to a duration of the startup latencies; and

adding together the startup latencies for jobs executing on individual computing units of the plurality of computing units to calculate aggregate latencies for the individual computing units.

7. The method of claim 6 , wherein the weight increases linearly according to the duration of the startup latencies.

8. The method of claim 6 , wherein the weight increases exponentially according to the duration of the startup latencies.

9. The method of claim 1 , wherein selecting the computing unit further comprises identifying the computing unit with the lowest aggregate latency that satisfies a resource constraint included in the request, wherein the resource constraint specifies a minimum amount of system components for execution of the job.

10. The method of claim 1 , wherein selecting the computing unit further comprises identifying the computing unit with the lowest aggregate latency that satisfies a predetermined failure constraint specifying an acceptable maximum latency-to-runtime ratio for the plurality of computing units.

11. The method of claim 1 , wherein the job includes one or both of (i) a pod process and (ii) a service process.

12. A system comprising:

a processor; and

a memory storing instructions which, when executed by the processor, cause the processor to:

receive a request to provision one or more system components for execution of a job;

calculate aggregate latencies of a plurality of computing units based on startup latencies for jobs executing on the plurality of computing units;

select a computing unit from among the plurality of computing units based on the aggregate latencies; and

provision system components from the computing unit.

13. The system of claim 12 , wherein calculating the aggregate latencies of the plurality of computing units further comprises:

receiving load metrics specifying at least a time to job availability for each job executing on the computing unit.

14. The system of claim 13 , wherein the startup latencies include the time to job availability.

15. The system of claim 12 , wherein the startup latencies further include a time to network connection.

16. The system of claim 15 , wherein the memory stores further instructions which, when executed by the processor when determining the time to network connection, cause the processor to:

transmit a network message into a network port of the job;

listen to the network port of the job until a reply is received; and

calculate the time to network connection as the time from when the network message is transmitted until the reply is received.

17. The system of claim 12 , wherein the memory stores further instructions which, when executed by the process when calculating the aggregate latencies, cause the processor to:

weight the startup latencies for jobs executing on the plurality of computing units with a weight that varies according to a duration of the startup latencies; and

add together the startup latencies for jobs executing on individual computing units of the plurality of computing units to calculate aggregate latencies for the individual computing units.

18. The system of claim 12 , wherein selecting the computing unit further comprises identifying the computing unit with the lowest aggregate latency that satisfies a resource constraint included in the request, wherein the resource constraint specifies a minimum amount of system components for execution of the job.

19. The system of claim 12 , wherein selecting the computing unit further comprises identifying the computing unit with the lowest aggregate latency that satisfies a predetermined failure constraint specifying an acceptable maximum latency-to-runtime ratio for the plurality of computing units.

20. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause the processor to:

receive a request to provision one or more system components for execution of a job;

calculate aggregate latencies of a plurality of computing units based on startup latencies for jobs executing on the plurality of computing units;

select a computing unit from among the plurality of computing units based on the aggregate latencies; and

provision system components from the computing unit.