NUMA aware TEP groups
Some embodiments provide a novel method for forwarding data messages between first and second host computers. To send, to a first machine of the first host, a second flow from a second machine of the second host in response to a first flow from the first machine, the method identifies from a set of tunnel endpoints (TEPs) of the first host a TEP that is a source TEP of the first flow. The method uses the identified TEP to identify one non-uniform memory access (NUMA) node of a set of NUMA nodes of the first host as the NUMA node associated with the first flow. The method selects, from a subset of TEPs of the first host that is associated with the identified NUMA node, one TEP as a destination TEP of the second flow. The method sends the second flow to the selected TEP of the first host.
1 . A method for forwarding data messages between first and second host computers, the method comprising:
to send, to a first machine executing on the first host computer, a second data message flow from a second machine executing on the second host computer that is in response to a first data message flow from the first machine:
identifying, from a plurality of tunnel endpoints (TEPs) of the first host computer, a particular TEP that is a source TEP of the first data message flow;
using the particular TEP to identify one particular non-uniform memory access (NUMA) node of a plurality of NUMA nodes of the first host computer as a NUMA node associated with the first data message flow;
selecting, from a subset of TEPs of the first host computer that is associated with the particular NUMA node, one TEP as a destination TEP of the second data message flow; and
sending the second data message flow to the selected TEP of the first host computer.
2 . The method of claim 1 , wherein identifying the particular TEP that is the source TEP of the first data message flow comprises identifying a particular TEP identifier (ID) identifying the particular TEP in an encapsulating header of at least one data message of the first data message flow.
3 . The method of claim 2 , wherein the encapsulating header is a first encapsulating header and the particular TEP ID is a first TEP ID, and sending the second data message flow to the selected TEP of the first host computer comprises encapsulating data messages of the second data message flow with a second encapsulating header specifying a second TEP ID identifying the selected TEP.
4 . The method of claim 3 , wherein identifying one particular NUMA node as the NUMA node associated with the first data message flow comprises performing a lookup operation in a mapping table to match the identified particular TEP ID to a particular NUMA node ID identifying the particular NUMA node.
5 . The method of claim 4 , wherein the mapping table stores, for each NUMA node of the first host computer, a set of one or more NUMA node ID to TEP ID mappings, including a particular set of mappings that map the particular NUMA node ID to a particular set of TEP IDs associated with the subset of TEPs.
6 . The method of claim 5 , wherein the mapping table is received from a set of one or more controllers that configures a software-defined network including the first and second host computers.
7 . The method of claim 4 , wherein the mapping table is a first mapping table, the method further comprising, after performing the lookup operation, storing, in a second mapping table, an association between a source network address of the first data message flow and the particular NUMA node ID to associate the first machine with the particular NUMA node.
8 . The method of claim 7 , wherein the lookup operation is a first lookup operation, and selecting the one TEP as the destination TEP of the second data message flow comprises:
performing a second lookup operation in the second mapping table to match a destination network address of the second data message flow to the particular NUMA node ID;
performing a third lookup operation in the first mapping table to match the particular NUMA node ID to the particular set of TEP IDs; and
selecting the one TEP from the subset of TEPs associated with the particular set of TEP IDs.
9 . The method of claim 8 , wherein selecting the one TEP from the subset of TEPs associated with the particular set of TEP IDs comprises performing a load balancing operation to select the one TEP from the subset of TEPs.
10 . The method of claim 8 , wherein the selected TEP is the particular TEP that is the source TEP of the first data message flow.
11 . The method of claim 8 , wherein the particular TEP that is the source TEP of the first data message flow is a first TEP associated with the particular NUMA node, and the selected TEP is a second TEP associated with the particular NUMA node.
12 . The method of claim 8 , wherein the source network address of the first data message flow and the destination network address of the second data message flow are a media access control (MAC) address of the first machine.
13 . The method of claim 1 , wherein the plurality of TEPs of the first host computer is a TEP group (TEPG).
14 . The method of claim 1 , wherein each NUMA node of the first host computer comprises a local memory and a set of processors that can provide data messages destined to other NUMA nodes to the other NUMA nodes.
15 . The method of claim 1 , wherein each NUMA node of the first host computer comprises a local memory and a set of processors that can access data from local memories of other NUMA nodes.
16 . The method of claim 1 , wherein each NUMA node is connected to a different set of one or more physical network interface cards (PNICs) to connect to the second host computer.
17 . The method of claim 16 , wherein each PNIC connects to the second host computer through one or more Top-of-Rack (ToR) switches.
18 . The method of claim 1 , wherein the first and second machines are first and second virtual machines (VMs).
19 . The method of claim 1 , wherein the identifying, selecting, and sending are performed by a virtual switch of the second host computer.
20 . A non-transitory machine readable medium storing a program for execution by at least one processing unit for forwarding data messages between first and second host computers, the program comprising sets of instructions for:
to send, to a first machine executing on the first host computer, a second data message flow from a second machine executing on the second host computer that is in response to a first data message flow from the first machine:
identifying, from a plurality of tunnel endpoints (TEPs) of the first host computer, a particular TEP that is a source TEP of the first data message flow;
using the particular TEP to identify one particular non-uniform memory access (NUMA) node of a plurality of NUMA nodes of the first host computer as a NUMA node associated with the first data message flow;
selecting, from a subset of TEPs of the first host computer that is associated with the particular NUMA node, one TEP as a destination TEP of the second data message flow; and
sending the second data message flow to the selected TEP of the first host computer.