Self-learning service scheduler for smart NICs
An example method comprises determining, by an edge services controller, based on a respective predicted resource utilization value for each of a plurality of servers, a corresponding server weight for each of the plurality of servers; the plurality of servers comprising respective network interface cards (NICs), wherein each NIC of the plurality of NICs comprises an embedded switch and a processing unit coupled to the embedded switch; determining, by the edge services controller, based on a respective predicted resource utilization value for each of a plurality of services, a corresponding application weight for each of the plurality of services; and scheduling, by the edge services controller, based on the respective server weight for a server of the plurality of servers and the respective application weight for the service, a service of the plurality of services on the server.
1. A method comprising:
predicting, by a controller for a plurality of servers, a resource utilization value for a service at a future time;
for each server of the plurality of servers, by the controller, predicting a central processing unit (CPU) utilization at the future time and a data processing unit (DPU) utilization at the future time, wherein the DPU utilization is for a DPU of a network interface card (NIC) of the server; and
scheduling, by the controller, based on the resource utilization value for the service at the future time, the CPU utilization at the future time for a particular server of the plurality of servers, and the DPU utilization at the future time for the DPU of the NIC of the particular server, the service to execute at least in part on the DPU of the NIC of the particular server.
2. The method of claim 1 , further comprising:
receiving, by the controller, historical usage data for the plurality of servers and a plurality of services, the plurality of services comprising the service; and
processing, by the controller, the historical usage data to train a machine learning model.
3. The method of claim 2 , further comprising:
predicting, by the trained machine learning model, the resource utilization value for the service at the future time; and
predicting, by the trained machine learning model, the DPU utilization at the future time for the DPU of the NIC of the particular server.
4. The method of claim 3 , further comprising:
iteratively training, by the controller, the trained machine learning model with a current DPU utilization for the DPU of the NIC of each server of the plurality of servers and with a current resource utilization value for each service of the plurality of services.
5. The method of claim 1 , wherein predicting the DPU utilization at the future time comprises processing, by the controller with a machine learning model, resource usage telemetry of a service instance for the service.
6. The method of claim 1 , wherein predicting the resource utilization value for the service at the future time comprises processing, by the controller with a machine learning model, usage telemetry data relative to a start time for the service.
7. The method of claim 1 , wherein predicting the resource utilization value for the service at the future time comprises:
receiving, by the controller, historical usage data associated with the service, a DPU usage, and one or more of:
a CPU usage;
a memory usage; or
a network usage; and
predicting, by the controller, the resource utilization value for the service at the future time as a function of the historical usage data.
8. The method of claim 7 , wherein the service is associated with a first timestamp corresponding to a start of the service, the method further comprising:
applying, by the controller, a vector autoregression machine learning model to the historical usage data to generate respective values, for a second timestamp following the first timestamp, for the DPU usage and one or more of:
the CPU usage;
the memory usage; or
the network usage.
9. The method of claim 1 , wherein scheduling the service to execute at least in part on the DPU of the NIC of the particular server comprises:
scheduling, by the controller, a first instance of the service to execute on the DPU of the NIC of the particular server; and
scheduling, by the controller, a second instance of the service on the DPU of the NIC of another server of the plurality of servers.
10. The method of claim 1 , wherein the DPU utilization at the future time for the DPU of the NIC of the server comprises an indication of availability of one or more of a processing core or a memory of the DPU of the NIC of the server.
11. A controller comprising processing circuitry having access to memory, the memory comprising instructions that, when executed, cause the processing circuitry to:
predict a resource utilization value for a service at a future time;
for each server of a plurality of servers, predict a central processing unit (CPU) utilization at the future time and a data processing unit (DPU) utilization at the future time, wherein the DPU utilization is for a DPU of a network interface card (NIC) of the server; and
schedule, based on the resource utilization value for the service at the future time, the CPU utilization at the future time for a particular server of the plurality of servers, and the DPU utilization at the future time for the DPU of the NIC of the particular server, the service to execute at least in part on the DPU of the NIC of the particular server.
12. The controller of claim 11 , wherein the instructions cause the processing circuitry to:
receive historical usage data for the plurality of servers and a plurality of services, the plurality of services comprising the service; and
process the historical usage data to train a machine learning model.
13. The controller of claim 12 , wherein the instructions cause the processing circuitry to:
predict, with the trained machine learning model, the resource utilization value for the service at the future time; and
predict, with the trained machine learning model, the DPU utilization at the future time for the DPU of the NIC of the particular server.
14. The controller of claim 13 , wherein the instructions cause the processing circuitry to:
iteratively train the trained machine learning model with a current DPU utilization for the DPU of the NIC of each server of the plurality of servers and with a current resource utilization value for each service of the plurality of services.
15. The controller of claim 11 , wherein to predict the DPU utilization at the future time, the instructions cause the processing circuitry to:
process, with a machine learning model, resource usage telemetry of a service instance for the service.
16. The controller of claim 11 , wherein to predict the resource utilization value for the service at the future time, the instructions cause the processing circuitry to:
process, with a machine learning model, usage telemetry data relative to a start time for the service.
17. The controller of claim 11 , wherein to predict the resource utilization value for the service at the future time, the instructions cause the processing circuitry to:
receive historical usage data associated with the service, a DPU usage, and one or more of:
a CPU usage;
a memory usage; or
a network usage; and
predict the resource utilization value for the service at the future time as a function of the historical usage data.
18. The controller of claim 17 ,
wherein the service is associated with a first timestamp corresponding to a start of the service, and
wherein the instructions cause the processing circuitry to:
apply a vector autoregression machine learning model to the historical usage data to generate respective values, for a second timestamp following the first timestamp, for the DPU usage and one or more of:
the CPU usage;
the memory usage; or
the network usage.
19. The controller of claim 11 , wherein the DPU utilization at the future time for the DPU of the NIC of the server comprises an indication of availability of one or more of a processing core or a memory of the DPU of the NIC of the server.
20. Non-transitory computer-readable media comprising instructions that, when executed, cause processing circuitry of a controller to:
predict a resource utilization value for a service at a future time;
for each server of a plurality of servers, predict a central processing unit (CPU) utilization at the future time and a data processing unit (DPU) utilization at the future time, wherein the DPU utilization is for a DPU of a network interface card (NIC) of the server; and
schedule, based on the resource utilization value for the service at the future time, the CPU utilization at the future time for a particular server of the plurality of servers, and the DPU utilization at the future time for the DPU of the NIC of the particular server, the service to execute at least in part on the DPU of the NIC of the particular server.