Smart network interface card control plane for distributed machine learning workloads
In certain embodiments, a method includes receiving, at an interface of a Smart network interface card (SmartNIC) of a computing device, via a network, a network data unit; processing, by a data allocator of a SmartNIC subsystem of the SmartNIC, the network data unit to make a determination that data included in the network data unit is intended for processing by an accelerator of the computing device, wherein the accelerator is configured to execute a machine learning algorithm; storing, by the data allocator and based on the determination, the data in a local buffer of the SmartNIC subsystem; identifying, by the data allocator, a memory resource associated with the accelerator; and transferring the data from the local buffer to the memory resource.
1. A computing device, comprising:
one or more processors;
a Smart network interface card (SmartNIC) comprising a SmartNIC subsystem comprising a data allocator and a local buffer;
an accelerator; and
one or more non-transitory computer readable media storing instruction which, when executed by the SmartNIC subsystem, cause the SmartNIC subsystem to:
receive, at an interface of the SmartNIC, via a network, a network data unit;
process, by the data allocator, the network data unit to make a determination that data included in the network data unit is intended for processing by the accelerator, wherein the accelerator is configured to execute a machine learning algorithm;
store, by the data allocator and based on the determination, the data in the local buffer;
identify, by the data allocator, a memory resource associated with the accelerator;
transfer the data from the local buffer to the memory resource; and
the instructions further cause the accelerator to execute the machine learning algorithm using the data.
2. The computing device of claim 1 , wherein the instructions further cause the SmartNIC to:
obtain, before receiving the network data unit, accelerator configuration information from the one or more processors.
3. The computing device of claim 2 , wherein the instructions further cause the SmartNIC to be configured to execute a bin packing algorithm.
4. The computing device of claim 2 , wherein the instructions further cause the SmartNIC to be configured to request that the one or more processors execute a bin packing algorithm.
5. The computing device of claim 2 , wherein the instructions further cause the SmartNIC to:
obtain, before receiving the network data unit, accelerator service rate information corresponding to the accelerator.
6. The computing device of claim 2 , wherein the accelerator configuration information includes accelerator virtualization information comprising identification of a plurality of accelerator slices of the accelerator.
7. The computing device of claim 6 , wherein the memory resource corresponds to an accelerator slice of the plurality of accelerator slices,
alternative heuristic algorithm.
8. The computing device of claim 2 , wherein the instructions further cause the SmartNIC to be configured to execute an alternative heuristic algorithm.
9. A method, comprising:
receiving, at an interface of a Smart network interface card (SmartNIC) of a computing device, via a network, a network data unit;
processing, by a data allocator of a SmartNIC subsystem of the SmartNIC, the network data unit to make a determination that data included in the network data unit is intended for processing by an accelerator of the computing device, wherein the accelerator is configured to execute a machine learning algorithm;
storing, by the data allocator and based on the determination, the data in a local buffer of the SmartNIC subsystem;
identifying, by the data allocator, a memory resource associated with the accelerator; and
transferring the data from the local buffer to the memory resource.
10. The method of claim 9 , further comprising
obtaining, before receiving the network data unit, accelerator configuration information from one or more processors of the computing device.
11. The method of claim 10 , further comprising configuring the SmartNIC to execute a bin packing algorithm.
12. The method of claim 10 , further comprising configuring the SmartNIC to request that the one or more processors execute a bin packing algorithm.
13. The method of claim 10 , further comprising:
obtaining, by the SmartNIC and before receiving the network data unit, accelerator service rate information corresponding to the accelerator.
14. The method of claim 10 , wherein the accelerator configuration information includes accelerator virtualization information comprising identification of a plurality of accelerator slices of the accelerator.
15. The method of claim 14 , wherein the memory resource corresponds to an accelerator slice of the plurality of accelerator slices.
16. A non-transitory computer-readable medium storing programming for execution by one or more processing units of a Smart network interface card (SmartNIC), the programming comprising instructions to:
receive, at an interface of the SmartNIC of a computing device, via a network, a network data unit;
process, by a data allocator of a SmartNIC subsystem of the SmartNIC, the network data unit to make a determination that data included in the network data unit is intended for processing by an accelerator of the computing device, wherein the accelerator is configured to execute a machine learning algorithm;
store, by the data allocator and based on the determination, the data in a local buffer of the SmartNIC subsystem;
identify, by the data allocator, a memory resource associated with the accelerator; and
transfer the data from the local buffer to the memory resource.
17. The non-transitory computer-readable medium of claim 16 , wherein the programming comprises further instructions to:
obtain, before receiving the network data unit, accelerator configuration information from one or more processors of the computing device.
18. The non-transitory computer-readable medium of claim 17 , wherein the programming comprises further instructions to configure the SmartNIC to execute a bin packing algorithm.
19. The non-transitory computer-readable medium of claim 17 , wherein the programming comprises further instructions to:
obtain, by the SmartNIC and before receiving the network data unit, accelerator service rate information corresponding to the accelerator.
20. The non-transitory computer-readable medium of claim 17 , wherein the accelerator configuration information includes accelerator virtualization information comprising identification of a plurality of accelerator slices of the accelerator, and the memory resource corresponds to an accelerator slice of the plurality of accelerator slices.