IP Library › Granted Patent US 11,533,234

Granted Patent B2

US 11,533,234 · App. 17/039,565 · Granted Dec 20, 2022

Autonomous distributed workload and infrastructure scheduling

Inventors: Andrew Cencini (Austin, TX); Cole Malone Crawford (Austin, TX); Erick Daniszewski (Austin, TX)

Assignee: Vapor IO Inc.

H04L41/0893 G06F1/189 G06F1/206 G06F1/26 G06F9/5083 H04L41/044 H04L67/1008 H04L67/1012 H04L67/1023 H04L67/1034 H04L67/12

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,533,234
App. No.: 17/039,565
Granted: Dec 20, 2022
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Provided is a process of autonomous distributed workload and infrastructure scheduling based on physical telemetry data of a plurality of different data centers executing a plurality of different workload distributed applications on behalf of a plurality of different tenants.

Claims (178)

1. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations comprising:

obtaining, with one or more processors, a request to process data for a user computing device at an application program interface of an edge compute-cluster manager, wherein:

the edge compute-cluster manager orchestrates execution of workload distributed applications across a set of heterogeneous edge computing nodes comprising at least a first edge computing node and a second edge computing node having differing amounts of different types of computing resources,

the first edge computing node is co-located with a first cellular tower and the second edge computing node is co-located with a second cellular tower, and

each edge computing node of the set of edge computing nodes has a different network, network address, or port from other edge computing nodes in the set of edge computing nodes;

obtaining, with one or more processors, performance data from the set of edge computing nodes, the performance data including one or more values indicative of latency associated with respective ones of the edge computing nodes;

determining, with one or more processors, a first latency value indicative of time associated with servicing the request by the first edge computing node and a second latency value indicative of time associated with servicing the request by the second edge computing node based on the request and respective edge computing node performance data indicative of at least one of:

a time to transmit the request to an edge computing node,

time to distribute workload tasks to one or more workload distributed applications by which the request is serviced among available resources, or resources to be made available, of the edge computing node,

time to execute the workload tasks with the one or more workload distributed applications with the available resources, or the resources to be made available, to service the request at the edge computing node, and

time to send a response from the edge computing node to the user computing device; and

allocating, with one or more processors, computing resources of the first edge computing node or the second edge computing node to the one or more workload distributed applications for servicing the request based on the first latency value and the second latency value, wherein allocating comprises:

accessing a policy in a plurality of different policies that indicate how to allocate resources, the policy specifying for different tasks of a workload distributed application:

a sensitivity of a task to latency, and

a sensitivity of a task to co-location with other tasks;

allocating one or more first tasks of the one or more workload distributed applications to the first edge computing node or the second edge computing node; and

allocating one or more second task of the one or more workload distributed applications to a third edge computing node based on a sensitivity of the one or more second tasks to co-location with the one or more first tasks.

2. The medium of claim 1 , further comprising:

determining amounts of cross-computing node messaging between edge computing nodes to determine sensitivities of different tasks of a workload application to distribution among different edge computing nodes.

3. The medium of claim 2 , the operations further comprising:

determining whether the computing resources of the first edge computing node or the second edge computing node should be allocated to the one or more workload distributed applications for servicing the request based on the first latency value, the second latency value, and the policy.

4. The medium of claim 1 , further comprising:

obtaining physical telemetry data indicating attributes of a physical environment in which the respective edge computing nodes execute; and

obtaining logical telemetry data indicating attributes of central processing unit utilization, memory utilization, and storage utilization, wherein allocating the computing resources of the first edge computing node or the second edge computing node to the one or more workload distributed applications for servicing the request comprises:

allocating based on the physical telemetry data and the logical telemetry data corresponding to the respective edge computing nodes.

5. The medium of claim 4 , further comprising:

accessing the policy or another policy in the plurality of different policies that indicates how to allocate resources based on the physical telemetry data and the logical telemetry data, wherein the physical telemetry data includes at least one of the following:

current temperature of a computing component of an edge computing node;

current temperature of ambient air in an edge computing node;

current temperature of ambient air outside of an edge computing node;

current humidity inside or outside of an edge computing node;

predicted temperature of a computing component of an edge computing node;

predicted temperature of ambient air in an edge computing node;

predicted temperature of ambient air outside of an edge computing node;

predicted humidity inside or outside of an edge computing node;

geolocation of an edge computing node;

location of a rack in an edge computing node;

location of a computing device on a rack of an edge computing node;

current thermal load of a computing device of an edge computing node;

current thermal load of a rack of an edge computing node;

current thermal load of an edge computing node;

predicted thermal load of a computing device of an edge computing node;

predicted thermal load of a rack of an edge computing node; or

predicted thermal load of an edge computing node.

6. The medium of claim 1 , wherein:

the set of edge computing nodes includes instances of micro-kernels, containers, virtual machines, or computing devices of the respective edge computing nodes.

7. The medium of claim 1 , further comprising:

obtaining physical telemetry data indicating attributes of a physical environment in which the respective edge computing nodes execute; and

accessing the policy or another policy in the plurality of policies that indicates how to allocate computing resources of the set of edge computing nodes based on the physical telemetry data and the performance data.

8. The medium of claim 7 , the operations further comprising:

accessing a first rule in a set of rules of the policy or the another policy, wherein the first rule specifies:

a first threshold of thermal load on cooling systems of candidate edge computing nodes, and

a second threshold of difference in latency of a workload task between candidate edge computing nodes,

wherein computing resources of the first edge computing node are allocated to the workload task in response to a difference between the first latency value and the second latency value being below the second threshold and a thermal load of the first edge computing node being greater than the first threshold and less than a thermal load of the second edge computing node.

9. The medium of claim 8 , wherein:

each policy in the plurality of policies corresponds to a different tenant in a plurality of tenants,

a given policy in the plurality of policies is selected based on a tenant associated with the request, and

a workload corresponding to the tenant is isolated from workloads of other tenants.

10. The medium of claim 8 , wherein the policy or the another policy comprises:

a thermal model for respective ones of the edge computing nodes, the thermal model comprising a model of heat-rejection and a model of heat-generation within a respective edge computing node based on workload; and

determining cooling system loads by inputting the obtained physical telemetry data into at least some of the thermal models.

11. The medium of claim 1 ,

wherein latency values comprise one or more of a bandwidth utilization, packet loss rate, or network latency.

12. The medium of claim 1 , wherein the set of heterogenous edge computing nodes comprises:

at least some edge computing nodes that include tensor processing units and some other edge computing nodes that do not include tensor processing units, or

at least some edge computing nodes that include field programmable gate arrays and some other edge computing nodes that do not include tensor processing units.

13. The medium of claim 1 , wherein the set of heterogenous edge computing nodes comprises:

a first subset of edge computing nodes that include tensor processing units,

a second subset of edge computing nodes that include field programmable gate arrays, and

a third subset of edge computing nodes that include tensor processing units and field programmable gate arrays.

14. The medium of claim 13 , the operations further comprising:

detecting that the first edge computing node has become available;

determining whether the first edge computing node is categorized as including a first type of computing resource; and

distributing task data to the first edge computing node in response to a determination that the first edge computing node is categorized as including the first type of computing resource.

15. The medium of claim 13 , wherein:

each subset of edge computing nodes comprises at least one edge computing node of three or more edge computing nodes.

16. The medium of claim 13 ,

wherein:

each subset of edge computing nodes comprises at least one edge computing node of three or more edge computing nodes, and

each of the three edge computing nodes have different amounts of at least one type of computing resource.

17. The medium of claim 1 , wherein a latency value corresponding to an edge computing node is based on two or more of:

a time to transmit the request to two edge computing node,

time to send a response from the edge computing node to the user computing device.

18. The medium of claim 1 , wherein a latency value associated with an edge computing node is based on each of:

a time to transmit the request to the edge computing node,

time to send a response from the edge computing node to the user computing device.

19. A method comprising:

obtaining, with one or more processors, a request to process data for a user computing device at an application program interface of an edge compute-cluster manager, wherein:

the first edge computing node is co-located with a first cellular tower and the second edge computing node is co-located with a second cellular tower, and

each edge computing node of the set of edge computing nodes has a different network, network address, or port from other edge computing nodes in the set of edge computing nodes;

determining, with one or more processors, a first latency value indicative of response time associated with servicing the request by the first edge computing node and a second latency value indicative of response time associated with servicing the request by the second edge computing node based on the request and respective edge node performance data indicative of at least one of:

a time to transmit the request to an edge computing node,

time to send a response from the edge computing node to the user computing device; and

accessing a policy in a plurality of different policies that indicate how to allocate resources, the policy specifying for different tasks of a workload distributed application:

a sensitivity of a task to latency, and

a sensitivity of a task to co-location with other tasks;

allocating one or more first tasks of the one or more workload distributed applications to the first edge computing node or the second edge computing node; and

20. The method of claim 19 , further comprising:

21. The method of claim 20 , further comprising:

22. The method of claim 19 , further comprising:

obtaining physical telemetry data indicating attributes of a physical environment in which the respective edge computing nodes execute; and

allocating based on the physical telemetry data and the logical telemetry data corresponding to the respective edge computing nodes.

23. The method of claim 22 , further comprising: