Efficient scatter-gather over an uplink
A network interface device is connected to a host computer by having a memory controller, and a scatter-gather offload engine linked to the memory controller. The network interface device prepares a descriptor including a plurality of specified memory locations in the host computer, incorporates the descriptor in exactly one upload packet, transmits the upload packet to the scatter-gather offload engine via the uplink, invokes the scatter-gather offload engine to perform memory access operations cooperatively with the memory controller at the specified memory locations of the descriptor, and accepts results of the memory access operations.
1. A method of communication, comprising the steps of:
in a network interface device connected to a host computer by an uplink, the host computer comprising a memory, a memory controller, and a scatter-gather offload engine linked to the memory controller;
determining that a group of scatter-gather operations are smaller than a threshold data size and performing the following for the group of operations that are smaller than the threshold data size:
preparing a descriptor comprising a plurality of specified memory locations for the group of operations that are smaller than the threshold data size;
incorporating the descriptor in an upload packet;
transmitting the upload packet from the network interface device to the scatter-gather offload engine via the uplink;
invoking the scatter-gather offload engine to perform memory access operations cooperatively with the memory controller at the specified memory locations of the descriptor; and
returning results of the memory access operations to the network interface device.
2. The method according to claim 1 , wherein the memory access operations comprise a gather operation, and the results comprise stored data that is read from the specified memory locations.
3. The method according to claim 1 , wherein the memory access operations comprise a scatter operation, and the descriptor further comprises data to be stored at respective ones of the specified memory locations.
4. The method according to claim 1 , wherein returning results comprises transmitting the results from the host computer to the network interface device in a response packet via the uplink.
5. The method according to claim 1 , wherein preparing a descriptor comprises generating respective pointers to the specified memory locations.
6. The method according to claim 1 , wherein preparing a descriptor comprises specifying a stride that defines an offset between successive ones of the specified memory locations.
7. The method according to claim 6 , wherein preparing a descriptor further comprises incorporating a base address of the memory therein, and specifying respective sizes of data segments at the specified memory locations.
8. The method according to claim 7 , wherein preparing a descriptor further comprises specifying a total size of the data segments at the specified memory locations and a number of times to repeat a pattern defined by the stride and an offset between successive instances of the pattern.
9. A network interface device connected to a host computer external to the network interface device by an uplink, the host computer comprising a memory, a memory controller, and a scatter-gather offload engine linked to the memory controller, the network interface device being operative for:
determining that a group of scatter-gather operations are smaller than a threshold data size and performing the following for the group of operations that are smaller than the threshold data size:
preparing a descriptor comprising, for the group of operations that are smaller than the threshold data size, a plurality of specified memory locations of the host computer;
incorporating the descriptor in an upload packet;
transmitting the upload packet from the network interface device to the scatter-gather offload engine via the uplink;
invoking the scatter-gather offload engine to perform memory access operations cooperatively with the memory controller at the specified memory locations of the descriptor; and
accepting results of the memory access operations via the uplink.
10. The network interface device according to claim 9 , wherein the memory access operations comprise a gather operation, and the results comprise stored data that is read from the specified memory locations.
11. The network interface device according to claim 9 , the memory access operations comprise a scatter operation, and the descriptor further comprises data to be stored at respective ones of the specified memory locations.
12. The network interface device according to claim 9 , wherein accepting results comprises accepting the results from the host computer in the network interface device in a response packet via the uplink.
13. The network interface device according to claim 9 , wherein preparing a descriptor comprises generating respective pointers to the specified memory locations.
14. The network interface device according to claim 9 , wherein preparing a descriptor comprises specifying a stride that defines an offset between successive ones of the specified memory locations.
15. The network interface device according to claim 14 , wherein preparing a descriptor further comprises incorporating a base address of the memory therein, and specifying respective sizes of data segments at the specified memory locations.
16. The network interface device according to claim 15 , wherein preparing a descriptor further comprises specifying a total size of the data segments at the specified memory locations and a number of times to repeat a pattern defined by the stride and an offset between successive instances of the pattern.
17. The method according to claim 1 , wherein the upload packet comprises exactly one upload packet.
18. The network interface device according to claim 9 , wherein the upload packet comprises exactly one upload packet.
19. An apparatus for communication, comprising:
a network interface device; and
a host computer connected to the network interface device by an uplink, the host computer comprising a memory, a memory controller, and a scatter-gather offload engine linked to the memory controller, wherein the network interface device is operative for:
preparing a descriptor comprising a plurality of specified memory locations of the host computer;
incorporating the descriptor in exactly one upload packet;
transmitting the upload packet from the network interface device to the scatter-gather offload engine via the uplink;
invoking the scatter-gather offload engine to perform memory access operations cooperatively with the memory controller at the specified memory locations of the descriptor; and
accepting results of the memory access operations via the uplink,
wherein preparing a descriptor comprises: specifying a stride that defines an offset between successive ones of the specified memory locations; incorporating a base address of the memory therein; specifying respective sizes of data segments at the specified memory locations; and specifying a total size of the data segments at the specified memory locations and a number of times to repeat a pattern defined by the stride and an offset between successive instances of the pattern.