Workload optimization system

1. A workload optimization system, comprising:

a workload manager subsystem that is configured to provide a plurality of workloads that are each configured to generate a job when executed, wherein the workload management system includes an agent registry that:

identifies a plurality of available agents that are each configured to execute a job having job requirements; and

includes agent registrations having:

agent information for each of the plurality of available agents that is configured for use in assigning a job to that agent; and

agent address information for each of the plurality of available agents that is configured for use in providing a job to that agent when that job is assigned to that agent;

an agent infrastructure subsystem that is coupled to the workload manager subsystem and that includes a first container host that is configured to provide a first container having a first agent that is included in the plurality of available agents and that is configured to process a job generated by at least one of the plurality of workloads, wherein the first agent facilitates communications of the job between the first container and the workload manager subsystem, and wherein the agent infrastructure subsystem registers agents with the workload manager subsystem; and

a workload resource optimization subsystem that is coupled to the workload manager subsystem and the agent infrastructure subsystem, wherein the workload resource optimization subsystem is configured to:

monitor a job queue of jobs that were generated by the plurality of workloads and that are to be processed by the first container via the first agent;

determine that the job queue satisfies a container generation condition and, in response, provide instructions to the first container host to retrieve a second container image from a container image cache, retrieve a second agent image from the container image cache, and use the second container image and the second agent image to generate a second container that includes a second agent;

communicate second agent information and second agent address information from the agent infrastructure subsystem to the workload manager subsystem for registration of the second container and the second agent in the agent registry;

monitor a first container host utilization of the first container host that is based at least in part on a container utilization of the first container and the second container; and

determine the first container host utilization of the first container host satisfies a container host activation condition that is based on a time required to activate a new container host before the first container host reaches a maximum container hosting capability and, in response, provide instructions to the agent infrastructure subsystem to activate a second container host that is configured to provide a third container having a third agent.

2. The workload optimization system of claim 1 , wherein the workload resource optimization subsystem is configured to:

monitor a second container host utilization of the second container host that is based at least in part on a container utilization of the third container; and

determine the first container host utilization of the first container host and the second container host utilization of the second container host satisfy a container host deactivation condition and, in response, provide container host instruction to the agent infrastructure subsystem to cause the agent infrastructure subsystem to:

transfer agents from the second container host to at least one container provided by the first container host; and

deactivate the second container host.

3. The workload optimization system of claim 1 , wherein the workload resource optimization subsystem is configured to:

determine that the job queue satisfies a container decommission condition and, in response, decommission at least one of the first container, the second container, and the third container.

4. The workload optimization system of claim 1 , wherein at least one of the first container host and the second container host is emulated by a virtual machine operated by a virtual machine engine that is included on a physical computing device.

5. The workload optimization system of claim 1 , wherein the workload manager subsystem includes a first workload manager and a second workload manager, and wherein the workload resource optimization subsystem is configured to:

monitor the job queue of jobs that were generated by the plurality of workloads on the first workload manager and the second workload manager that are to be processed by the first container and the second container; and

determine that the job queue of jobs that were generated by the plurality of workloads on the second workload manager satisfies the container generation condition and, in response, provide instructions to the second container host to generate a fourth container that includes a fourth agent and that is configured to process the jobs that were generated by the plurality of workloads on the second workload manager.

6. The workload optimization system of claim 1 , wherein the first agent and the second agent are grouped in a first agent pool that is configured to execute at least one job.

7. An information handling system (IHS), comprising:

a communication system;

a processor that is coupled to the communication system; and

a non-transitory memory that is coupled to the processor and that includes instructions that, when executed by the processor, cause the processor to provide a workload resource optimization engine that is configured to:

monitor job queue information for a job queue of jobs that are generated by a plurality of workloads and that are included on a workload manager subsystem that includes an agent registry that:

identifies a plurality of available agents that are each configured to execute a job having job requirements; and

includes agent registrations having:

agent information for each of the plurality of available agents that is configured for use in assigning a job to that agent; and

agent address information for each of the plurality of available agents that is configured for use in providing a job to that agent when that job is assigned to that agent;

monitor first container host utilization information for a first container host that is included on an agent infrastructure subsystem, that provides a first container having a first agent, and that processes a job generated by at least one of the plurality of workloads;

determine from the job queue information that the job queue satisfies a container generation condition and, in response, provide first instructions to the first container host to retrieve a second container image from a container image cache, retrieve a second agent image from the container image cache, and use the second container image and the second agent image to generate a second container that includes a second agent;

communicate second agent information and second agent address information from the agent infrastructure subsystem to the workload manager subsystem to cause the workload manager subsystem to register the second container and the second agent in the agent registry; and

determine, subsequent to the providing the first instructions to the first container host to generate the second container, that the first container host utilization information for the first container host satisfies a container host activation condition that is based on a time required to activate a new container host before the first container host reaches a maximum container hosting capability and, in response, provide second instructions to the agent infrastructure subsystem to activate a second container host to provide a third container having a third agent.

8. The IHS of claim 7 , wherein the workload resource optimization engine is configured to:

monitor second container host utilization information of the second container host that is based at least in part on a container utilization of the third container;

determine the first container host utilization information of the first container host and the second container host utilization information of the second container host satisfy a container host deactivation condition and, in response, provide container host instruction to the agent infrastructure subsystem to cause the agent infrastructure subsystem to:

transfer agents from the second container host to at least one container provided by the first container host; and

deactivate the second container host.

9. The IHS of claim 7 , wherein the workload resource optimization engine is configured to:

determine that the job queue satisfies a container decommission condition and, in response, decommission at least one of the first container, the second container, and the third container.

10. The IHS of claim 7 , wherein at least one of the first container host and the second container host is emulated by a virtual machine operated by a virtual machine engine that is included on a physical computing device.

11. The IHS of claim 7 , wherein the workload manager subsystem includes a first workload manager and a second workload manager, and wherein the workload resource optimization engine is configured to:

monitor the job queue of jobs that were generated by the plurality of workloads on the first workload manager and the second workload manager and that are to be processed by the first container and the second container; and

determine that jobs that are included in the job queue of jobs and that were generated by the plurality of workloads on the second workload manager satisfy the container generation condition and, in response, provide third instructions to the second container host to generate a fourth container that includes a fourth agent and that is configured to process the jobs that were generated by the plurality of workloads on the second workload manager.

12. The IHS of claim 7 , wherein the first agent and the second agent are grouped in a first agent pool that is configured to execute at least one job.

13. A method for workload resource optimization, comprising:

monitoring, by a workload resource optimization subsystem, a job queue of a plurality of jobs that are included on a workload manager subsystem and that are generated by at least one workload on the workload manager subsystem, wherein the workload manager subsystem includes an agent registry that:

identifies a plurality of available agents that are each configured to execute a job having job requirements; and

includes agent registrations having:

agent information for each of the plurality of available agents that is configured for use in assigning a job to that agent; and

agent address information for each of the plurality of available agents that is configured for use in providing a job to that agent when that job is assigned to that agent;

monitoring, by the workload resource optimization subsystem, first container host utilization information for a first container host that is included on an agent infrastructure subsystem, that provides a first container having a first agent, and that processes a job generated by at least one of the plurality of workloads, wherein the first agent facilitates communications of the execution of the job between the first container and the workload manager subsystem;

determining, by the workload resource optimization subsystem, that the job queue satisfies a container generation condition and, in response, providing instructions to the first container host to retrieve a second container image from a container image cache, retrieve a second agent image from the container image cache, and use the second container image and the second agent image to generate a second container that includes a second agent;

communicating, by the workload resource optimization subsystem, second agent information and second agent address information from the agent infrastructure subsystem to the workload manager subsystem to cause the workload manager subsystem to register the second container and the second agent in the agent registry;

monitoring, by the workload resource optimization subsystem, second container host utilization information of the first container host that is based at least in part on a container utilization of the first container and the second container; and

determining, by the workload resource optimization subsystem, the first container host utilization of the first container host satisfies a container host activation condition that is based on a time required to activate a new container host before the first container host reaches a maximum container hosting capability and, in response, providing instructions to the agent infrastructure subsystem to activate a second container host that is configured to provide a third container having a third agent.

14. The method of claim 13 , further comprising:

monitoring, by the workload resource optimization subsystem, a second container host utilization of the second container host that is based at least in part on a container utilization of the third container;

determining, by the workload resource optimization subsystem, the first container host utilization of the first container host and the second container host utilization of the second container host satisfy a container host deactivation condition and, in response, providing container host instruction to the agent infrastructure subsystem to cause the agent infrastructure subsystem to:

transfer, by the workload resource optimization subsystem, agents from the second container host to at least one container provided by the first container host; and

deactivate, by the workload resource optimization subsystem, the second container host.

15. The method of claim 13 , further comprising:

determining, by the workload resource optimization subsystem, that the job queue satisfies a container decommission condition and, in response, decommissioning at least one of the first container, the second container, and the third container.

16. The method of claim 13 , wherein at least one of the first container host and the second container host is emulated by a virtual machine operated by a virtual machine engine that is included on a physical computing device.

17. The method of claim 13 , further comprising:

monitoring, by the workload resource optimization subsystem, the job queue of jobs that were generated by the plurality of workloads on a first workload manager and a second workload manager that are to be processed by the first container and the second container, wherein the workload manager subsystem includes the first workload manager having a first application programming interface (API) and the second workload manager having a second API; and

determining, by the workload resource optimization subsystem, that the job queue of jobs that were generated by the plurality of workloads on the second workload manager satisfies the container generation condition and, in response, providing instructions to the second container host to generate a fourth container that includes a fourth agent and that is configured to process the jobs that were generated by the plurality of workloads on the second workload manager.