Pre-emptive container load-balancing, auto-scaling and placement
A resource usage platform is disclosed. The platform performs preemptive container load balancing, auto scaling, and placement in a computing system. Resource usage data is collected from containers and used to train a model that generates inferences regarding resource usage. The resource usage operations are performed based on the inferences and on environment data such as available resources, service needs, and hardware requirements.
1. A method, comprising:
receiving a request to invoke a container in a computing system from a container image;
generating an inference from an inference service by providing an input to the inference service, wherein the inference includes a prediction of resources needed for the request;
receiving environment data associated with the computing system, the environment data including information about resources available at nodes in the computing system;
attaching services information to the container that describe services used by the container;
scheduling the container to run on a node in the computing system based on the inference and the environment data and based on the services information attached to the container, wherein the node has access to the services; and
running the container on the node.
2. The method of claim 1 , wherein the environment data comprises container metadata including hardware requirements, resource usage data, and service requirements.
3. The method of claim 1 , further comprising training a model with telemetry data collected from running containers in the computing system.
4. The method of claim 3 , further comprising collecting telemetry data from each container on each node in the computing system.
5. The method of claim 1 , wherein the environment data includes resources consumed by each container at each node, resources available at each node, and execution times of each container.
6. The method of claim 1 , further comprising scheduling the request in an existing and running container, scheduling the request in a new container on a selected node, or rejecting the request.
7. The method of claim 1 , further comprising training a model using telemetry data and/or aggregating the telemetry data.
8. The method of claim 1 , further comprising auto scaling containers in the computing system and/or performing federating learning.
9. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
receiving a request to invoke a container in a computing system from a container image;
generating an inference from an inference service by providing an input to the inference service, wherein the inference includes a prediction of resources needed for the request;
receiving environment data associated with the computing system, the environment data including information about resources available at nodes in the computing system;
attaching services information to the container that describe services used by the container;
scheduling the container to run on a node in the computing system based on the inference and the environment data and based on the services information attached to the container, wherein the node has access to the services; and
running the container on the node.
10. The non-transitory storage medium of claim 9 , wherein the environment data comprises container metadata including hardware requirements, resource usage data, and service requirements.
11. The non-transitory storage medium of claim 9 , further comprising training a model with telemetry data collected from running containers in the computing system.
12. The non-transitory storage medium of claim 11 , further comprising collecting telemetry data from each container on each node in the computing system.
13. The non-transitory storage medium of claim 9 , wherein the environment data includes resources consumed by each container at each node, resources available at each node, and execution times of each container.
14. The non-transitory storage medium of claim 9 , further comprising scheduling the request in an existing and running container, scheduling the request in a new container on a selected node, or rejecting the request.
15. The non-transitory storage medium of claim 9 , further comprising training a model using telemetry data and/or aggregating the telemetry data.
16. The non-transitory storage medium of claim 9 , further comprising auto scaling containers in the computing system and/or performing federating learning.