Virtual machine (VM) migration with smart network interface cards (NICs)
Some embodiments provide a novel method of migrating a particular virtual machine (VM) from a first host computer to a second host computer. The first host computer of some embodiments has a physical network interface card (PNIC) that performs at least one of network forwarding operations and middlebox service operations for the particular VM. The first host computer sends, to the PNIC of the first host computer, a request for state information relating to at least one of network forwarding operations and middlebox service operations that the PNIC performs for the particular VM. The first host computer receives the state information from the PNIC. The first host computer provides the state information received from the PNIC to the second host computer as part of a data migration that is performed to migrate the particular VM from the first host computer to the second host computer.
1 . A method of migrating a first virtual machine (VM) from a first host computer to a second host computer, the first host computer having a first physical network interface card (PNIC) that includes (i) one or more software components configured to perform network forwarding operations and middlebox service operations for the first VM by executing on a processor of the first PNIC and (ii) one or more hardware accelerator devices separate from the processor of the first PNIC that are also configured to perform the network forwarding operations and the middlebox service operations for the first VM, the method comprising:
at the first host computer:
sending, to the first PNIC of the first host computer, one or more requests for (i) state information relating to the network forwarding operations that the one or more software components perform for the first VM and (ii) state information relating to the middlebox service operations that the one or more software components perform for the first VM;
receiving the network forwarding and middlebox service state information from the first PNIC; and
sending the network forwarding and middlebox service state information received from the first PNIC to the second host computer as part of a data migration that is performed to migrate the first VM from the first host computer to the second host computer.
2 . The method of claim 1 , wherein sending the one or more requests comprises sending a first request for the network forwarding state information and sending a second request for the middlebox service state information.
3 . The method of claim 2 , wherein:
the first request is sent by a VM migration module of the first host computer to a host forwarding module of the first host computer which forwards the first request to the first PNIC, the host forwarding module including a virtual switch of the first VM, and
the second request is sent by the VM migration module to a host service module of the first host computer which provides the second request to the first PNIC, the host service module being configured to intercept data messages of the first VM.
4 . The method of claim 1 , further comprising:
before sending the one or more requests, directing the first PNIC to disable service hardware offload to cause the one or more software components to begin performing the middlebox service operations for the first VM instead of the one or more hardware accelerator devices.
5 . The method of claim 1 , wherein the middlebox service state information comprises at least one of distributed firewall state information, load balancing state information, and network address translation (NAT) state information.
6 . The method of claim 1 , wherein the network forwarding operations for the first VM comprise at least one of layer 2 (L2) forwarding operations and layer 3 (L3) forwarding operations.
7 . The method of claim 1 , further comprising:
populating, on the first host computer, configuration state information relating to a configuration of the first VM; and
sending, to the second host computer, the configuration state information along with the network forwarding and middlebox service state information received from the first PNIC.
8 . The method of claim 1 , wherein the network forwarding and middlebox service state information received from the first PNIC is sent to a VM migration module executing on the second host computer, the VM migration module executing on the second host computer being configured to migrate the first VM from the first host computer to the second host computer.
9 . The method of claim 8 , wherein the network forwarding and middlebox service state information is sent to the VM migration module executing on the second host computer by a VM migration module executing on the first host computer, the VM migration modules executing on the first and second host computers being configured to migrate the first VM from the first host computer to the second host computer.
10 . The method of claim 1 , wherein the second host computer has a second PNIC configured to perform network forwarding operations and middlebox service operations for a set of VMs operating on the second host computer.
11 . The method of claim 10 , wherein after the first VM is migrated to the second host computer, the second PNIC performs the network forwarding operations and the middlebox service operations for the first VM.
12 . The method of claim 11 , wherein after the first VM is migrated to the second host computer, the first PNIC deletes the network forwarding and middlebox service state information for the first VM.
13 . The method of claim 1 , further comprising:
receiving, by the first host computer from a third host computer, state information for a second VM that has been requested to be migrated from the third host computer to the first host computer; and
providing, by the first host computer, the received state information for the second VM to the first PNIC, wherein the first PNIC then performs network forwarding operations and middlebox service operations for the second VM.
14 . The method of claim 13 , wherein the third host computer includes a second PNIC, and the third host computer received the state information for the second VM from the second PNIC.
15 . A non-transitory machine readable medium storing a program that, when executed by at least one processing unit of a first host computer, causes the first host computer to perform a method of migrating a first virtual machine (VM) from the first host computer to a second host computer, the first host computer having a first physical network interface card (PNIC) that includes (i) one or more software components configured to perform network forwarding operations and middlebox service operations for the first VM by executing on a processor of the first PNIC and (ii) one or more hardware accelerator devices separate from the processor of the first PNIC that are also configured to perform the network forwarding operations and the middlebox service operations for the first VM, the program comprising sets of instructions for:
at the first host computer:
sending, to the first PNIC of the first host computer, one or more requests for (i) state information relating to the network forwarding operations that the one or more software components perform for the first VM and (ii) state information relating to the middlebox service operations that the one or more software components perform for the first VM;
receiving the network forwarding and middlebox service state information from the first PNIC; and
sending the network forwarding and middlebox service state information received from the first PNIC to the second host computer as part of a data migration that is performed to migrate the first VM from the first host computer to the second host computer.
16 . The non-transitory machine readable medium of claim 15 , wherein sending the one or more requests comprises sending a first request for the network forwarding state information and sending a second request for the middlebox service state information.
17 . The non-transitory machine readable medium of claim 16 , wherein:
the first request is sent by a VM migration module of the first host computer to a host forwarding module of the first host computer which provides the first request to the first PNIC, the host forwarding module including a virtual switch of the first VM, and
the second request is sent by the VM migration module to a host service module of the first host computer which provides the second request to the first PNIC, the host service module being configured to intercept data messages of the first VM.
18 . The non-transitory machine readable medium of claim 15 , wherein the method further comprises:
before sending the one or more requests, directing the first PNIC to disable service hardware offload to cause the one or more software components to begin performing the middlebox service operations for the first VM instead of the one or more hardware accelerator devices.
19 . The non-transitory machine readable medium of claim 15 , wherein the middlebox service state information comprises at least one of distributed firewall state information, load balancing state information, and network address translation (NAT) state information.
20 . The non-transitory machine readable medium of claim 15 , wherein the network forwarding operations for the first VM comprise at least one of layer 2 (L2) forwarding operations and layer 3 (L3) forwarding operations.