Network interface card, storage apparatus, and packet receiving method and sending method
This application discloses a network interface card so as to improve reliability of a storage apparatus using an RDMA network and improve storage access performance of a single QP connection. A network interface card includes a parser and at least one shared connector, where the parser is configured to distribute a received first remote direct memory access RDMA packet to a first shared connector in the at least one shared connector; and the first shared connector is configured to establish a QP connection to a peer network device, and determine a first queue pair QP corresponding to a first controller based on the first RDMA packet, so as to implement communication with the first controller, where the first shared connector is separately connected to a plurality of controllers including the first controller, and one QP corresponding to each of the plurality of controllers corresponds to the first shared connector.
1. A network interface card (NIC), comprising:
a parser; and
at least one shared connector, wherein:
the parser is configured to distribute a received first remote direct memory access (RDMA) packet to a first shared connector of the at least one shared connector; and
the NIC is configured to establish a queue pair (QP) connection to a peer network device via the first shared connector, and determine a first QP corresponding to a first controller based on the first RDMA packet, so as to implement communication with the first controller, wherein the first shared connector is separately connected to a plurality of controllers comprising the first controller, and one QP corresponding to each of the plurality of controllers is corresponding to the first shared connector.
2. The network interface card according to claim 1 , wherein the first shared connector is further configured to:
if the first RDMA packet is not an acknowledgment packet, store the first RDMA packet to a storage space corresponding to the first QP.
3. The network interface card according to claim 1 , wherein the first shared connector is further configured to:
if the first RDMA packet is a tail packet, a single packet, or an acknowledgment packet, post a completion queue entry (CQE) to a completion queue (CQ) associated with the first QP, wherein the CQE is used to indicate that an RDMA operation corresponding to the first RDMA packet is completed.
4. The network interface card according to claim 1 , wherein the first shared connector is specifically configured to:
if the first RDMA packet is an initial packet or a single packet of a SEND message, determine the first QP based on information in a non-volatile memory express (NVMe) submission queue entry (SQE) in the first RDMA packet; wherein
the information in the NVMe SQE is corresponding to the first controller.
5. The network interface card according to claim 1 , wherein the first shared connector is specifically configured to:
if the first RDMA packet is an acknowledgment packet of an RDMA WRITE message, an acknowledgment packet of a SEND message, or an initial packet or a single packet of an RDMA READ response message, determine the first QP based on a packet sequence number of the first RDMA packet and a correspondence between a packet sequence number of a second RDMA packet and the first QP; wherein
the second RDMA packet is an RDMA packet sent by the first QP; the packet sequence number of the first RDMA packet is corresponding to the packet sequence number of the second RDMA packet; when the first RDMA packet is the acknowledgment packet of the RDMA WRITE message, the second RDMA packet belongs to an RDMA WRITE message; when the first RDMA packet is the acknowledgment packet of the SEND message, the second RDMA packet belongs to a SEND message; and when the first RDMA packet is the initial packet or the single packet of the RDMA READ response message, the second RDMA packet belongs to an RDMA READ request message.
6. The network interface card according to claim 4 , wherein the first shared connector is further configured to:
if the first RDMA packet is an initial packet, store a correspondence between an initial packet of a message to which the first RDMA packet belongs and the first QP.
7. The network interface card according to claim 6 , wherein the first shared connector is further configured to:
if the first RDMA packet is a tail packet or a middle packet, determine an initial packet corresponding to the first RDMA packet based on a monotonically-increasing property of packet sequence numbers of segmented RDMA packets and the packet sequence number of the first RDMA packet; and
determine the first QP based on the initial packet corresponding to the first RDMA packet and the correspondence.
8. A network interface card, comprising a scheduler and at least one shared connector, wherein
the scheduler is configured to select, from a plurality of queue pairs QPs of a plurality of controllers in position, a second QP that is corresponding to a second shared connector and has a message to be sent, wherein the second shared connector is one of the at least one shared connector;
the second shared connector is configured to establish a QP connection to a peer network device, and send, to a host client, a message in a storage space corresponding to the second QP in a form of a second remote direct memory access RDMA packet; and the second shared connector is separately connected to the plurality of controllers, and one QP corresponding to each of the plurality of controllers is corresponding to the second shared connector.
9. The network interface card according to claim 8 , wherein the second shared connector is further configured to:
if the second RDMA packet belongs to an RDMA WRITE message, a SEND message, or an RDMA READ request message, store a correspondence between a packet sequence number of the second RDMA packet and the second QP.
10. The network interface card according to claim 8 , wherein the scheduler is specifically configured to:
select the second QP from a plurality of QPs corresponding to the second shared connector by using round robin.
11. A packet receiving method, applied to the network interface card according to claim 1 , wherein the method comprises:
distributing, by a parser, a received first remote direct memory access RDMA packet to a first shared connector in at least one shared connector; and
establishing, by the first shared connector, a QP connection to a peer network device, and determining a first queue pair QP corresponding to a first controller based on the first RDMA packet, so as to implement communication with the first controller, wherein the first shared connector is separately connected to a plurality of controllers comprising the first controller, and one QP corresponding to each of the plurality of controllers is corresponding to the first shared connector.
12. The method according to claim 11 , further comprising:
if the first RDMA packet is not an acknowledgment packet, storing, by the first shared connector, the first RDMA packet to a storage space corresponding to the first QP.
13. The method according to claim 11 , further comprising:
if the first RDMA packet is a tail packet, a single packet, or an acknowledgment packet, posting, by the first shared connector, a completion queue entry CQE to a completion queue CQ associated with the first QP, wherein the CQE is used to indicate that an RDMA operation corresponding to the first RDMA packet is completed.
14. The method according to claim 11 , wherein the determining a first queue pair QP corresponding to a first controller based on the first RDMA packet comprises:
if the first RDMA packet is an initial packet or a single packet of a SEND message, determining, by the first shared connector, the first QP based on information in a non-volatile memory express (NVMe) submission queue entry (SQE) in the first RDMA packet; wherein
the information in the NVMe SQE is corresponding to the first controller.
15. The method according to claim 11 , wherein the determining a first queue pair QP corresponding to a first controller based on the first RDMA packet comprises:
if the first RDMA packet is an acknowledgment packet of an RDMA WRITE message, an acknowledgment packet of a SEND message, or an initial packet or a single packet of an RDMA READ response message, determining, by the first shared connector, the first QP based on a packet sequence number of the first RDMA packet and a correspondence between a packet sequence number of a second RDMA packet and the first QP; wherein
the second RDMA packet is an RDMA packet sent by the first QP; the packet sequence number of the first RDMA packet is corresponding to the packet sequence number of the second RDMA packet; when the first RDMA packet is the acknowledgment packet of the RDMA WRITE message, the second RDMA packet belongs to an RDMA WRITE message; when the first RDMA packet is the acknowledgment packet of the SEND message, the second RDMA packet belongs to a SEND message; and when the first RDMA packet is the initial packet or the single packet of the RDMA READ response message, the second RDMA packet belongs to an RDMA READ request message.
16. The method according to claim 14 , further comprising:
if the first RDMA packet is an initial packet, storing, by the first shared connector, a correspondence between an initial packet of a message to which the first RDMA packet belongs and the first QP.
17. The method according to claim 16 , further comprising:
if the first RDMA packet is a tail packet or a middle packet, determining, by the first shared connector, an initial packet corresponding to the first RDMA packet based on a monotonically-increasing property of packet sequence numbers of segmented RDMA packets and the packet sequence number of the first RDMA packet; and
determining the first QP based on the initial packet corresponding to the first RDMA packet and the correspondence.
18. A packet sending method, applied to the network interface card according to claim 8 , wherein the method comprises:
selecting, by a scheduler from a plurality of queue pairs QPs of a plurality of controllers in position, a second QP that is corresponding to a second shared connector and has a message to be sent, wherein the second shared connector is one of at least one shared connector; and
establishing, by the second shared connector, a QP connection to a peer network device, and sending, to a host client, a message in a storage space corresponding to the second QP in a form of a second remote direct memory access RDMA packet; wherein the second shared connector is separately connected to the plurality of controllers, and one QP corresponding to each of the plurality of controllers is corresponding to the second shared connector.
19. The method according to claim 18 , further comprising:
if the second RDMA packet belongs to an RDMA WRITE message, a SEND message, or an RDMA READ request message, storing, by the second shared connector, a correspondence between a packet sequence number of the second RDMA packet and the second QP.
20. A storage apparatus, comprising a network interface card (NIC) and a plurality of controllers, wherein the network interface card is separately connected to each of the plurality of controllers, and the network interface card comprise a parser and at least one shared connector, wherein
the parser is configured to distribute a received first RDMA packet to a first shared connector in the at least one shared connector; and
the storage apparatus is configured to establish a QP connection to a peer network device via the first shared connector, and determine a first QP corresponding to a first controller based on the first RDMA packet, so as to implement communication with the first controller, wherein the first shared connector is separately connected to a plurality of controllers comprising the first controller, and one QP corresponding to each of the plurality of controllers is corresponding to the first shared connector.