State sharing between smart NICs
Some embodiments provide a method for synchronizing state between multiple smart NICs of a host computer that perform operations using dynamic state information. At a first smart NIC of the plurality of smart NICs, the method stores a set of dynamic state information. The method synchronizes the set of dynamic state information across a communication channel that connects the smart NICs so that each of the smart NICs also stores the set of dynamic state information.
1. A method for synchronizing state between a plurality of smart network interface controllers (NICs) of a host computer that perform operations using dynamic state information, the method comprising:
at a first smart NIC of the plurality of smart NICs of the host computer, storing a set of dynamic connection state information that the first smart NIC uses to perform stateful processing of data messages belonging to a plurality of ongoing connections between data compute machines executing on the host computer and other endpoints external to the host computer, the dynamic connection state information comprising connection tracking data, the connection tracking data comprising a list of active transport layer connections; and
synchronizing the set of dynamic connection state information to at least a second smart NIC of the plurality of smart NICs of the host computer, across a communication channel that connects the plurality of smart NICs of the host computer, so that each of the second smart NIG also stores the set of dynamic connection state information, wherein the second smart NIC uses the dynamic connection state to perform stateful processing of data messages belonging to at least a subset of the ongoing connections between the machines executing on the host computer and other endpoints when the first smart NIC fails;
wherein the connection tracking data further comprises congestion windows for each of at least a subset of the active transport layer connections, wherein synchronizing the set of dynamic connection state information allows the second smart NIC to use the congestion window for a particular ongoing connection to perform stateful processing of data messages belonging to the particular connection rather than requiring the particular connection to be re-established and the congestion window for the particular connection reset.
2. The method of claim 1 , wherein each of the smart NICs of the host computer executes a smart NIC operating system that performs virtual networking operations, including the stateful processing operations, on data messages sent to and from the data compute machines executing on the host computer using the connection tracking data.
3. The method of claim 1 , wherein the virtual stateful processing comprises performing firewall operations that determine whether to allow the data messages.
4. The method of claim 1 , wherein the dynamic connection state information comprises tracked storage connection information for performing virtualized storage operations.
5. The method of claim 1 , wherein:
each respective smart NIC in the plurality of smart NICs of the host computer performs stateful processing operations using a respective set of dynamic connection state information;
the plurality of ongoing connections is a first plurality of ongoing connections and the set of dynamic connection state information is a first set of dynamic connections state information; and
the method further comprises, upon failure of the second smart NIC, performing stateful processing operations previously performed by the second smart NIC for a second plurality of onging connections between the data compute machines executing on the host computer and other external endpoints by using a second set of dynamic connection state information received from the second smart NIC in a synchronization operation.
6. The method of claim 1 , wherein the communication channel is a private communication channel with high bandwidth and low latency.
7. The method of claim 6 , wherein the private communication channel is a set of physical cables that only carries communication between the smart NICs.
8. The method of claim 7 , wherein the set of physical cables comprises, for each respective pair of smart NICs in the plurality of smart NICs, a respective physical cable connecting the pair of smart NICs.
9. The method of claim 7 , wherein each smart NIC in the plurality of smart NICs is directly connected to two other smart NICs via the set of physical cables.
10. The method of claim 7 , wherein the set of physical cables connects to a set of one or more physical switches to carry communication between the smart NICs.
11. The method of claim 6 , wherein:
the plurality of smart NICs of the host computer are connected via physical ports to a physical network in order to transmit data to and receive data from other host computers; and
the private communication channel is a logically private communication channel that utilizes the physical network.
12. The method of claim 11 , wherein the logically private communication channel is an overlay network.
13. The method of claim 6 , wherein the private communication channel uses a Peripheral Component Interconnect Express (PCie) subsystem of the host computer.
14. A non-transitory machine-readable medium storing a program which when executed by at least one processing unit of a first smart network interface controller (NIC) of a host computer synchronizes state between a plurality of smart network interface controllers (NICs) of host computer that perform operations using dynamic state information, the program comprising sets of instructions for:
storing a set of dynamic connection state information at the first smart NIC of the host computer that the first smart NIC uses to perform stateful processing of data messages belonging to a plurality of ongoing connections between data compute machines executing on the host computer and other endpoints external to the host computer, the dynamic connection state information comprising connection tracking data; and
synchronizing the set of dynamic connection state information to at least a second smart NIC of the plurality of smart NICs of the host computer, across a communication channel that connects the plurality of smart NICs of the host computer, so the second NIC also stores the set of dynamic connection state information, wherein the second smart NIC uses the dynamic connection state to perform stateful processing of data messages belonging to at least a subset of the ongoing connections between the machines executing on the host computer and other endpoints when the first smart NIC fails;
wherein each of the smart NICs of the host computer executes a smart NIC operating system that performs virtual networking operations, including the stateful processing operations, on data messages sent to and from the data compute machines executing on the host computer using the connection tracking data;
wherein the connection tracking data comprises (i) a list of active transport layer connections and (ii) congestion windows for each of at least a subset of the active transport layer connections, wherein synchronizing the set of dynamic connection state information allows the second smart NIC to use the congestion window for a particular ongoing connection to perform stateful processing of data messages belonging to the particular connection rather than requiring the particular connection to be re-established and the congestion window for the particular connection reset.
15. The non-transitory machine-readable medium of claim 14 , wherein:
each respective smart NIC in the plurality of smart NICs of the host computer performs stateful processing operations using a respective set of dynamic connection state information;
the plurality of ongoing connections is a first plurality of ongoing connections and the set of dynamic connection state information is a first set of dynamic connections state information; and
the program further comprises a set of instructions for, upon failure of the second smart NIC, performing stateful processing operations previously performed by the second smart NIC for a second plurality of onging connections between the data compute machines executing on the host computer and other external endpoints by using a second set of dynamic connection state information received from the second smart NIC in a synchronization operation.
16. The non-transitory machine-readable medium of claim 14 , wherein:
the communication channel is a set of physical cables that only carries communication between the smart NICs; and
the set of physical cables comprises, for each respective pair of smart NICs in the plurality of smart NICs, a respective physical cable connecting the pair of smart NICs.
17. The non-transitory machine-readable medium of claim 14 , wherein:
the communication channel is a set of physical cables that only carries communication between the smart NICs; and
the set of physical cables connects to a set of one or more physical switches to carry communication between the smart NICs.
18. The non-transitory machine-readable medium of claim 14 , wherein:
the smart NICs are connected via physical ports to a physical network in order to transmit data to and receive data from other host computers; and
the communication channel is a logically private communication channel that utilizes the physical network.
19. The non-transitory machine-readable medium of claim 14 , wherein the communication channel uses a Peripheral Component Interconnect Express (PCie) subsystem of the host computer.