Autonomous distributed workload and infrastructure scheduling
Provided is a process of autonomous distributed workload and infrastructure scheduling based on physical telemetry data of a plurality of different data centers executing a plurality of different workload distributed applications on behalf of a plurality of different tenants.
1. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations comprising:
obtaining, at a data center, data for processing by an application executed at the data center from a computing device of a user via a wireless network, wherein:
the data center provides edge-based computing services to the computing device of the user,
the application is associated with a first tenant of a plurality of tenants of the data center, and
the data center comprises a computing resource, wherein access to the computing resource is isolated to the first tenant;
obtaining, with one or more processors, physical telemetry data of the data center;
accessing, with one or more processors, a policy of a plurality of policies that indicates how to allocate computing resources based on the association between the application and the first tenant, wherein the policy specifies a set of resource allocation actions for the application and comprises a set of weights by which attributes of the physical telemetry data and execution of the application are combined to determine a plurality of weighted scores, each weighted score being associated with a different candidate resource allocation action of the set of resource allocation actions;
allocating, with one or more processors, the computing resource to the application based on the policy and the physical telemetry data, wherein allocating based on the policy and the physical telemetry data comprises selecting a resource allocation action based on the plurality of weighted scores;
executing, with one or more processors, an operation of the application using the computing resource to determine a computed result; and
sending, with one or more processors, the computed result to the computing device of the user via the wireless network.
2. The medium of claim 1 , wherein allocating the computing resource comprises selecting the data center from among a plurality of data centers, and wherein a set of computing resources of the plurality of data centers are in communication with one another.
3. The medium of claim 2 , wherein the plurality of data centers comprises more than 1,000 data centers.
4. The medium of claim 2 , wherein each respective data center of the plurality of data centers executes a respective instance of the application.
5. The medium of claim 2 , wherein the data center is an edge-based computing facility, and wherein the edge-based computing facility is a shared data center environment, and wherein the plurality of data centers vary in size.
6. The medium of claim 2 , where the data center is co-located with a cellular tower, and wherein each respective data center of the plurality of data centers is co-located with a respective cellular tower.
7. The medium of claim 1 , wherein the data center is within cellular range of the computing device.
8. The medium of claim 1 , wherein the data center is an edge data center.
9. The medium of claim 1 , wherein executing the operation of the application comprises executing a machine learning operation to determine the computed result.
10. The medium of claim 1 , wherein:
the computing device of the user is a self-driving automobile or an autonomous drone,
the data comprises image data, and
the computed result comprises a classification.
11. The medium of claim 1 , wherein:
the computing resource is a first computing resource,
the policy associates a first latency value to the first computing resource,
the policy associates a second latency value to a second computing resource, and
allocating the first computing resource comprises selecting the first computing resource based on an association between the application and the first latency value.
12. The medium of claim 1 , wherein the data center is a first data center, the operations further comprising:
storing a value of the data on a persistent memory of a first computing device of the first data center; and
sending the value to a persistent memory of a second computing device of a second data center amongst a plurality of data centers based on a determination that the second computing device is operating as a leader node, wherein:
the plurality of data centers comprises the first data center, the second data center, and a third data center;
the leader node is elected a based on a set of votes provided by computing devices of the plurality of data centers, and
the leader node distributes the value to a third computing device of the third data center of the plurality of data centers.
13. The medium of claim 12 , wherein the second computing device distributes a command to the first computing device, and wherein the command comprises an update to the plurality of policies.
14. The medium of claim 1 , wherein the physical telemetry data comprises a temperature and humidity of the data center.
15. The medium of claim 1 , wherein allocating the computing resource comprises:
searching a parameter space to determine a response value based on a series of operations to minimize or maximize an objective function, wherein the parameter space comprises a parameter causing the allocation of the computing resource; and
allocating the computing resource based on the response value.
16. The medium of claim 15 , wherein searching the parameter space comprises:
obtaining a neural network configured based on the objective function; and
determining the response value using the neural network.
17. The medium of claim 1 , wherein the set of weights further comprises one or more weights corresponding to performance metrics comprising at least one of temperature, processor utilization, fan speed, memory utilization, bandwidth utilization, packet loss, storage utilization, or power utilization.
18. The medium of claim 1 , the operations further comprising:
obtaining resource metadata, wherein the resource metadata comprises a location of a second data center;
selecting, with a scheduling algorithm, a computing resource of the second data center based on the location; and
executing the application using the computing resource of the second data center.
19. The medium of claim 1 , wherein at least some other policies in the plurality of policies are each associated with different tenant accounts of the plurality of tenants.
20. The medium of claim 1 , wherein allocating the computing resource comprises:
determining whether the data center satisfies a criterion that the data center be one of a set of data center types; and
allocating the computing resource in response to a determination that the data center satisfies the criterion.
21. The medium of claim 1 , the operations further comprising obtaining a latency value associated with the application, wherein allocating the computing resource comprises allocating the computing resource based on the latency value.
22. The medium of claim 1 , wherein the computing resource is a first computing resource, and wherein allocating the first computing resource based on the policy comprises allocating the first computing resource based on a network latency between the first computing resource and a second computing resource of a second data center.
23. The medium of claim 1 , the operations further comprising steps for managing rack-mounted computing devices.
24. The medium of claim 1 , wherein allocating the computing resource comprises steps for allocating the computing resource based on the physical telemetry data.
25. The medium of claim 1 , wherein attributes of execution of the application comprises at least one of a predicted processor utilization, memory utilization, network utilization, and power consumption associated with executing the operation of the application.
26. A method comprising:
obtaining, at a data center, data for processing by an application executed at the data center from a computing device of a user via a wireless network, wherein:
the data center provides edge-based computing services to the computing device of the user,
the application is associated with a first tenant of a plurality of tenants of the data center, and
the data center comprises a computing resource, wherein access to the computing resource is isolated to the first tenant;
obtaining, with one or more processors, physical telemetry data of the data center;
accessing, with one or more processors, a policy of a plurality of policies that indicates how to allocate computing resources based on the association between the application and the first tenant, wherein the policy specifies a set of resource allocation actions for the application and comprises a set of weights by which attributes of the physical telemetry data and execution of the application are combined to determine a plurality of weighted scores, each weighted score being associated with a different candidate resource allocation action of the set of resource allocation actions;
allocating, with one or more processors, the computing resource to the application based on the policy and the physical telemetry data, wherein allocating based on the policy and the physical telemetry data comprises selecting a resource allocation action based on the plurality of weighted scores;
executing, with one or more processors, an operation of the application using the computing resource to determine a computed result; and
sending, with one or more processors, the computed result to the computing device of the user via the wireless network.
27. A computer-implemented method, the computer implemented method comprising:
obtaining, at a data center, data for processing, by an application executed at the data center or another data center in a plurality of data centers, from a computing device of a user via a wireless network, wherein:
the data center provides edge-based computing services to the computing device of the user,
the data center is co-located with a cellular tower, and wherein each other respective data center of the plurality of data centers is co-located with a respective cellular tower,
the application is associated with a first tenant of a plurality of tenants of the data center, and
the data center comprises a computing resource, wherein access to the computing resource is isolated to the first tenant;
obtaining, with one or more processors, physical telemetry data comprising at least a temperature and humidity of the data center;
accessing, with one or more processors, a policy of a plurality of policies that indicates how to allocate computing resources based on the association between the application and the first tenant, wherein the policy specifies a set of resource allocation actions;
allocating, with one or more processors, the computing resource to the application based on the policy and the physical telemetry data, wherein:
the policy associates a first latency value to a first computing resource,
the policy associates a second latency value to a second computing resource, and
the first computing resource is selected as the computing resource for allocation to the application based on an association between the application and the first latency value;
executing, with one or more processors, an operation of the application using the computing resource to determine a computed result; and
sending, with one or more processors, the computed result to the computing device of the user via the wireless network.