Throttling for bandwidth imbalanced data transfers
Techniques are disclosed to throttle bandwidth imbalanced data transfers. In some examples, an example computer-implemented method may include splitting a payload of a data transfer operation over a network fabric into multiple chunk get operations, starting the execution of a threshold number of the chunk get operations, and scheduling the remaining chunk get operations for subsequent execution. The method may also include executing a scheduled chunk get operation in response determining a completion of an executing chunk get operation. In some embodiments, the chunk get operations may be implemented as triggered operations.
1. A system to throttle bandwidth imbalanced data transfers between a sender node and a receiver node coupled via a network fabric, the system comprising:
one or more non-transitory machine readable mediums configured to store executable computer program instructions; and
one or more processors configured to execute the computer program instructions stored on the one or more non-transitory machine readable mediums, execution of the computer program instructions being configured to cause the one or more processors to:
choose a number of chunk get operations based on a write bandwidth of non-volatile memory of the receiver node, the write bandwidth of the non-volatile memory being less than a network bandwidth of the network fabric;
split a payload data transfer operation over the network fabric into a plurality of chunk get operations, wherein a chunk get operation of the plurality of chunk get operations transfers a chunk of a payload over the network fabric via a remote direct memory access (RDMA) read operation executed by the receiver node;
execute a threshold number, c, of chunk get operations from the plurality of chunk get operations;
schedule remaining chunk get operations from the plurality of chunk get operations as scheduled chunk get operations for subsequent execution; and
in response to a determination of a completion of an executing chunk get operation and a determination of at least one scheduled chunk get operation, execute a chunk get operation from the scheduled chunk get operations.
2. The system of claim 1 , wherein the payload data transfer operation includes one or more of a transfer operation over the network fabric, a write operation to the non-volatile memory, and a transfer operation in a high performance computing system.
3. The system of claim 1 , wherein to split the payload data transfer operation is based on a capability of the network fabric.
4. The system of claim 1 , wherein the threshold number, c, is based on a capability of the network fabric.
5. The system of claim 1 , wherein each chunk get operation of the plurality of chunk get operations is a triggered operation.
6. The system of claim 5 , wherein the triggered operation includes a hardware implementation.
7. The system of claim 1 , wherein the scheduled chunk get operations are executed in sequence.
8. A computer-implemented method to throttle bandwidth imbalanced data transfers between a sender node and a receiver node coupled via a network fabric, the method comprising:
choosing a number of chunk get operations based on a write bandwidth of non-volatile memory of the receiver node, the write bandwidth of the non-volatile memory being less than a network bandwidth of the network fabric;
splitting a payload data transfer operation over the network fabric into a plurality of chunk get operations, wherein a chunk get operation of the plurality of chunk get operations transfers a chunk of a payload over the network fabric via a remote direct memory access (RDMA) read operation executed by the receiver node;
executing a threshold number, c, of chunk get operations from the plurality of chunk get operations;
scheduling remaining chunk get operations from the plurality of chunk get operations as scheduled chunk get operations for subsequent execution; and
in response to a determination of a completion of an executing chunk get operation and a determination of at least one scheduled chunk get operation, executing a chunk get operation from the scheduled chunk get operations.
9. The method of claim 8 , wherein the payload data transfer operation includes a write operation to the non-volatile memory.
10. The method of claim 8 , wherein the payload data transfer operation is conducted in a high performance computing system.
11. The method of claim 8 , wherein splitting the payload data transfer operation is based on a capability of the network fabric.
12. The method of claim 8 , wherein the threshold number, c, is based on a capability of the network fabric.
13. The method of claim 8 , wherein each chunk get operation of the plurality of chunk get operations is a triggered operation.
14. The method of claim 13 , wherein the triggered operation is based on a counter.
15. The method of claim 13 , wherein the triggered operation includes a hardware implementation.
16. A computer program product including one or more non-transitory machine readable mediums encoded with executable computer program instructions that when executed by one or more processors cause a process to be carried out to throttle bandwidth imbalanced data transfers between a sender node and a receiver node coupled via a network fabric, the process comprising:
choosing a number of chunk get operations based on a write bandwidth of non-volatile memory of the receiver node, the write bandwidth of the non-volatile memory being less than a network bandwidth of the network fabric;
splitting a payload data transfer operation over the network fabric into a plurality of chunk get operations, wherein a chunk get operation of the plurality of chunk get operations transfers a chunk of a payload over the network fabric via a remote direct memory access (RDMA) read operation executed by the receiver node;
executing a threshold number, c, of chunk get operations from the plurality of chunk get operations;
scheduling remaining chunk get operations from the plurality of chunk get operations as scheduled chunk get operations for subsequent execution; and
in response to a determination of a completion of an executing chunk get operation and a determination of at least one scheduled chunk get operation, executing a chunk get operation from the scheduled chunk get operations.
17. The computer program product of claim 16 , wherein the payload data transfer operation includes one or more of a transfer operation over the network fabric, a write operation to the non-volatile memory, and a transfer operation in a high performance computing system.
18. The computer program product of claim 16 , wherein the payload data transfer operation includes a write operation to a non-volatile memory.
19. The computer program product of claim 16 , wherein splitting the payload data transfer operation is based on capability of the network fabric.
20. The computer program product of claim 16 , wherein the threshold number, c, is based on capability of the network fabric.