Virtual machine traffic engineering
A mechanism to allow switch manager software to determine bandwidth consumption and cooperate with a VM manager. Counter hardware measures network traffic between the various ports of the switch stack. The switch manager determines the MAC or IP addresses of devices connected to each port, which are provided to the VM manager to correlate VMs to ports. The switch manger collects statistics from the counter hardware to determine traffic flow levels between the various ports. A list of high traffic port pairs is provided to the VM manager. In conjunction with each port is a list of available ports, identified by at least one MAC or IP address associated with that port, having capacity to receive the traffic of the respective ports of each port pair and provided in order of least to most hops, to allow the VM manager to select a closer port to receive a migrated VM.
1. A network comprising:
a virtual machine (VM) manager for managing the operations of a plurality of VMs on a plurality of physical servers, at least some VMs communicating with at least some VMs on different servers;
a network switch arrangement having a plurality of ports for interconnecting the plurality of physical servers which measures packet flow for each port and for each ingress and egress port pair where VMs on different physical servers are communicating; and
a switch manager cooperating with the network switch arrangement and coupled to said VM manager,
wherein said switch manager
maps packet addresses to ports of said network switch arrangement,
provides such mapping to said VM manager,
receives values indicating used bandwidth between a plurality of port pairs,
determines available bandwidth for ports of said network switch arrangement interconnecting the plurality of physical servers,
determines ports having available bandwidth greater than the used bandwidth for at least a portion of said plurality of port pairs,
for each of the ports in said at least a portion of said plurality of port pairs organizes a list of ports having available bandwidth greater than the used bandwidth for the port pair that are closer to a port in the port pair than the other port in the port pair in a closer to farther order, such that said organized list only includes ports having available bandwidth greater than the used bandwidth for a given port pair, for each port in each port pair of said at least a portion of said plurality of port pairs, and
provides the organized list with closer port information to said VM manager, and
wherein said VM manager
receives said mapping of packet addresses to ports,
maps ports to VMs of the plurality of VMs using said mapping of packet address to ports,
receives said organized list with closer port information,
determines current physical servers for VMs of the plurality of VMs,
determines that a first VM communicating with a second VM on a different physical server and desired to be moved cannot be moved to be on the same server as the second VM,
selects a different server for the first VM based on said organized list with closer port information and
moves the first VM to the selected different server so that the first VM is closer to the second VM after being moved.
2. The network of claim 1 , wherein the packet addresses are media access control (MAC) or Internet Protocol (IP) addresses.
3. The network of claim 1 , wherein said switch manager includes the media access control (MAC) or Internet Protocol (IP) address associated with each port in said organized list of closer port information.
4. The network of claim 1 , wherein said VM manager determining that a first VM cannot be moved includes determining that both the first and second VMs cannot fit on either physical server of the first VM and the second VM.
5. The network of claim 1 , wherein said VM manager determining that a first VM cannot be moved includes determining that a physical server that can fit both the first VM and the second VM is not available.
6. A network switch arrangement comprising:
a plurality of ports for interconnecting a plurality of physical servers, the plurality of physical servers including a plurality of virtual machines (VMs), at least some VMs communicating with at least some VMs on different servers, which measure packet flow for each port and for each ingress and egress port pair where VMs on different servers are communicating; and
a switch manager coupled to said plurality of ports for communicating with a VM manager,
wherein said switch manager
maps packet addresses to ports of said network switch arrangement,
provides such mapping to said VM manager,
receives values indicating used bandwidth between a plurality of port pairs,
determines available bandwidth for ports of said switch arrangement interconnecting the plurality of physical servers,
determines ports having available bandwidth greater than the used bandwidth for at least a portion of said plurality of port pairs,
for each of the ports in said at least a portion of said plurality of port pairs organizes a list of ports having available bandwidth greater than the used bandwidth for the port pair that are closer to a port in the port pair than the other port in the port pair in a closer to farther order, such that said organized list only includes ports having available bandwidth greater than the used bandwidth for a given port pair, for each port in each port pair of said at least a portion of said plurality of port pairs, and
provides the organized list with closer port information to said VM manager.
7. The network switch arrangement of claim 6 , wherein the packet addresses are media access control (MAC) or Internet Protocol (IP) addresses.
8. The network switch arrangement of claim 6 , wherein said switch manager includes the media access control (MAC) or Internet Protocol (IP) address associated with each port in said organized list of closer port information.
9. A computer comprising:
a processor;
a network interface for connecting to a network switch arrangement interconnecting a plurality of physical servers, the plurality of physical servers including a plurality of virtual machines (VMs), at least some VMs communicating with at least some VMs on different servers, the network switch arrangement having a plurality of ports for connection to the plurality of physical servers, where each port of the plurality of ports measures packet flow for each port and for each ingress and egress port pair where VMs on different servers are communicating, said network interface coupled to said processor;
memory for storing data and programs, said programs including a VM manager, wherein said VM manager
receives a mapping of packet addresses to ports,
maps ports to VMs of the plurality of VMs using said mapping of packet address to ports,
receives an organized list with closer port information, said organized list including a list of ports having available bandwidth greater than the used bandwidth for the port pair that are closer to a port in the port pair than the other port in the port pair in a closer to farther order, such that said organized list only includes ports having available bandwidth greater than the used bandwidth for a given port pair, for each port in each port pair of said at least a portion of said plurality of port pairs,
determines current physical servers for VMs of the plurality of VMs,
determines that a first VM communicating with a second VM on a different physical server and desired to be moved cannot be moved to be on the same server as the second VM,
selects a different server for the first VM based on said organized list with closer port information and
moves the first VM to the selected different server so that the first VM is closer to the second VM after being moved.
10. The computer of claim 9 , wherein the packet addresses are media access control (MAC) or Internet Protocol (IP) addresses.
11. The computer of claim 9 , wherein said VM manager determining that a first VM cannot be moved includes determining that both the first and second VMs cannot fit on either physical server of the first VM and the second VM.
12. The network of claim 9 , wherein said VM manager determining that a first VM cannot be moved includes determining that a physical server that can fit both the first VM and the second VM is not available.
13. A non-transitory computer readable storage medium or media having computer-executable instructions stored therein for an application which causes a processor to perform the following method, the method comprising:
operating a switch manager which:
maps packet addresses to ports of a network switch arrangement interconnecting a plurality of physical servers, the plurality of physical servers including a plurality of virtual machines (VMs), at least some VMs communicating with at least some VMs on different servers, the network switch arrangement having a plurality of ports for connection to the plurality of physical servers, where each port of the plurality of ports measures packet flow for each port and for each ingress and egress port pair where VMs on different servers are communicating;
provides such mapping to a VM manager;
receives values indicating used bandwidth between a plurality of port pairs;
determines available bandwidth for ports of said network switch arrangement interconnecting the plurality of physical servers;
determines ports having available bandwidth greater than the used bandwidth for at least a portion of said plurality of port pairs;
for each of the ports in said at least a portion of said plurality of port pairs organizes a list of ports having available bandwidth greater than the used bandwidth for the port pair that are closer to a port in the port pair than the other port in the port pair in a closer to farther order, such that said organized list only includes ports having available bandwidth greater than the used bandwidth for a given port pair, for each port in each port pair of said at least a portion of said plurality of port pairs; and
provides the organized list with closer port information to the VM manager.
14. The non-transitory computer readable storage medium or media of claim 13 , wherein the packet addresses are media access control (MAC) or Internet Protocol (IP) addresses.
15. The non-transitory computer readable storage medium or media of claim 13 , wherein said switch manager includes the media access control (MAC) or Internet Protocol (IP) address associated with each port in said organized list of closer port information.
16. A non-transitory computer readable storage medium or media having computer-executable instructions stored therein for an application which causes a processor to perform the following method, the method comprising:
operating a virtual machine (VM) manager which
receives a mapping of packet addresses to ports of a network switch arrangement interconnecting a plurality of physical servers, the plurality of physical servers including a plurality of VMs, at least some VMs communicating with at least some VMs on different servers, the switch arrangement having a plurality of ports for connection to the plurality of physical servers, where each port of the plurality of ports measures packet flow for each port and for each ingress and egress port pair where VMs on different servers are communicating;
maps ports to VMs of the plurality of VMs using said mapping of packet address to ports;
receives an organized list with closer port information, said organized list including a list of ports having available bandwidth greater than the used bandwidth for the port pair that are closer to a port in the port pair than the other port in the port pair in a closer to farther order, such that said organized list only includes ports having available bandwidth greater than the used bandwidth for a given port pair, for each port in each port pair of said at least a portion of said plurality of port pairs;
determines current physical servers for VMs of the plurality of VMs;
determines that a first VM communicating with a second VM on a different physical server and desired to be moved cannot be moved to be on the same server as the second VM;
selects a different server for the first VM based on said organized list with closer port information; and
moves the first VM to the selected different server so that the first VM is closer to the second VM after being moved.
17. The non-transitory computer readable storage medium or media of claim 16 , wherein the packet addresses are media access control (MAC) or Internet Protocol (IP) addresses.
18. The non-transitory computer readable storage medium or media of claim 16 , wherein said VM manager determining that a first VM cannot be moved includes determining that both the first and second VMs cannot fit on either physical server of the first VM and the second VM.
19. The non-transitory computer readable storage medium or media of claim 16 , wherein said VM manager determining that a first VM cannot be moved includes determining that a physical server that can fit both the first VM and the second VM is not available.
20. The non-transitory computer readable storage medium or media of claim 16 , the method further comprising:
operating a switch manager which:
maps packet addresses to ports of the network switch arrangement;
provides such mapping to the VM manager;
receives values indicating used bandwidth between a plurality of port pairs;
determines available bandwidth for ports of said switch arrangement interconnecting the plurality of physical servers;
determines ports having available bandwidth greater than the used bandwidth for at least a portion of said plurality of port pairs;
for each of the ports in said at least a portion of said plurality of port pairs organizes a list of ports having available bandwidth greater than the used bandwidth for the port pair that are closer to a port in the port pair than the other port in the port pair in a closer to farther order; and
provides the organized list with closer port information to the VM manager.