Directing queries to nodes of a cluster of a container orchestration platform distributed across a host system and a hardware accelerator of the host system

1. A method comprising:

receiving queries at a gateway of a cluster of a container orchestration platform, wherein the cluster includes a host system comprising a first processing resource and a hardware accelerator of the host system, the hardware accelerator comprising a second processing resource, wherein the host system comprising the first processing resource or the hardware accelerator comprising the second processing resource comprises a primary node of the cluster, and wherein the host system and the hardware accelerator comprise a plurality of worker nodes of the cluster;

distributing, at the gateway, the queries among the plurality of worker nodes of the host system and the hardware accelerator based on a queuing model that takes into consideration respective compute capacities of the first processing resource of the host system and the second processing resource of the hardware accelerator;

performing auto-scaling to run a quantity of instances of an application in the plurality of worker nodes; and

responsive to receipt of the queries, directing the queries to the plurality of worker nodes according to the distributing for processing, by the plurality of worker nodes, the quantity of instances of the application.

2. The method of claim 1 , wherein the hardware accelerator comprises a System on a Chip (SOC) based Smart Network Interface Card (SmartNIC).

3. The method of claim 1 , wherein the application comprises a key-value store or an analytics application.

4. The method of claim 1 , wherein the container orchestration platform comprises a Kubernetes container orchestration platform.

5. The method of claim 1 , further comprising:

monitoring a number of the queries received at the gateway for processing by the application;

maintaining historical data regarding queries over time; and

predicting, based on the number of the queries received at the gateway and the historical data, a future workload for the application, wherein the auto-scaling is based on the predicted future workload.

6. The method of claim 5 , wherein the auto-scaling comprises:

measuring a metric for the application in the cluster; and

scaling the quantity of instances of the application based on a relation of the measured metric to a target value of the metric, the scaling comprising increasing or decreasing the quantity of instances of the application.

7. The method of claim 6 , wherein the measuring of the metric and the predicting of the future workload are performed by the hardware accelerator.

8. The method of claim 6 , wherein the target value of the metric is according to a service level agreement (SLA).

9. The method of claim 5 , wherein the predicting of the future workload is performed by the hardware accelerator.

10. The method of claim 1 , wherein the auto-scaling of the quantity of instances of the application comprises increasing or decreasing a quantity of containers or pods in which the instances of the application are to execute.

11. The method of claim 1 , wherein the host system comprises a plurality of hardware accelerators, and wherein the plurality of hardware accelerators are to serve as a subset of the plurality of worker nodes.

12. The method of claim 1 , wherein the primary node comprises a scheduler to schedule workloads on the plurality of worker nodes.

13. A non-transitory machine readable medium comprising instructions that when executed cause a system to:

receive queries at an Application Programming Interface (API) gateway of a cluster of a container orchestration platform, wherein the cluster includes a host system comprising a first processing resource and one or more hardware accelerators comprising a second processing resource, wherein the host system comprising the first processing resource or a hardware accelerator of the one or more hardware accelerators comprising the second processing resource comprises a primary node of the cluster, and wherein the host system and the one or more hardware accelerators comprise a plurality of worker nodes of the cluster;

determine, by the API gateway, a distribution of the queries among the plurality of worker nodes of the host system and the one or more hardware accelerators based on a queuing model that takes into consideration respective compute capacities of the host system and the one or more hardware accelerators;

perform auto-scaling to run a quantity of instances of an application in the plurality of worker nodes; and

responsive to receipt of the queries, direct the queries to the plurality of worker nodes according to the distribution for processing, by the plurality of worker nodes, the quantity of instances of the application.

14. The non-transitory machine readable medium of claim 13 , wherein a hardware accelerator of the one or more hardware accelerators comprises a System on a Chip (SOC) based Smart Network Interface Card (SmartNIC.

15. The non-transitory machine readable medium of claim 13 , wherein the auto-scaling of the quantity of instances of the application comprises increasing or decreasing a quantity of containers or pods in which the instances of the application are to execute.

16. The non-transitory machine readable medium of claim 13 , wherein the instructions when executed cause the system to:

monitor a number of the queries received at the API gateway for processing by the application;

access historical data regarding queries over time; and

predict, based on the number of the queries received at the API gateway and the historical data, a future workload for the application, wherein the auto-scaling is based on the predicted future workload.

17. The non-transitory machine readable medium of claim 16 , wherein the instructions when executed cause the system to:

measure a metric for the application in the cluster; and

scale the quantity of instances of the application based on a relation of the measured metric to a target value of the metric.

18. A system comprising:

a cluster comprising a gateway and a host system comprising a processing resource and a plurality of hardware accelerators, wherein the host system or a hardware accelerator of the plurality of hardware accelerators comprises a primary node of the cluster, and the host system and the plurality of hardware accelerators comprise a plurality of worker nodes of the cluster,

wherein the gateway comprises a non-transitory storage medium comprising instructions executable at the gateway to:

receive queries at the gateway; and

determine a distribution of the queries among the plurality of worker nodes of the host system and the plurality of hardware accelerators based on a queuing model that takes into consideration respective compute capacities of the host system and the plurality of hardware accelerators,

wherein the host system is to perform auto-scaling to run a quantity of instances of an application in the plurality of worker nodes, and

wherein the queries are directed to the plurality of worker nodes according to the distribution for processing, by the plurality of worker nodes, the quantity of instances of the application.

19. The system of claim 18 , wherein a given hardware accelerator of the plurality of hardware accelerators is to:

monitor a number of the queries received at the gateway for processing by the application;

access historical data regarding queries over time; and

predict, based on the number of the queries received at the gateway and the historical data, a future workload for the application, wherein the auto-scaling is based on the predicted future workload.

20. The system of claim 19 , wherein the given hardware accelerator is to:

measure a metric for the application in the cluster,

wherein the host system is to perform the auto-scaling by scaling the quantity of instances of the application based on a relation of the measured metric to a target value of the metric.