IP Library › Granted Patent US 7,299,294
Granted Patent B1
US 7,299,294 · App. 09/548,188 · Granted Nov 20, 2007

Distributed traffic controller for network data

Assignee: EMC Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,299,294
App. No.
09/548,188
Granted
Nov 20, 2007
Kind
B1
Abstract

A distributed gateway for controlling computer network data traffic dynamically reconfigures traffic assignments among multiple gateway machines for increased network availability. If one of the distributed gateway machines becomes unavailable, traffic assignments are moved among the multiple machines such that network availability is substantially unchanged. The machines of the distributed gateway form a cluster and communicate with each other using a Group Membership protocol word such that automatic, dynamic traffic assignment reconfiguration occurs in response to machines being added and deleted from the cluster, with no loss in functionality for the gateway overall, in a process that is transparent to network users, thereby providing a distributed gateway functionality that is scalable. Operation of the distributed gateway remains consistent as machines are added and deleted from the cluster. A scalable, distributed, highly available, load balancing network gateway is thereby provided, having multiple machines that function as a front server layer between the network and a back-end server layer having multiple machines functioning as Web file servers, FTP servers, or other application servers. The front layer machines comprise a server cluster that performs fail-over and dynamic load balancing for both server layers.

Claims (93)

1. A data traffic controller for a computer network, the data traffic controller comprising:

a network interface that permits communication with a subnet over which network data is sent and received; and

a plurality of computers connected in a cluster between the network interface and the subnet to form a distributed gateway for communicating network data between said network interface and said subnet, each of which computers executes a distributed gateway application that dynamically assigns any of a plurality of virtual network addresses selected from a pool of virtual addresses to any of a plurality of primary network addresses to send network data to an intended host on the subnet, wherein network data intended for said host on the subnet is addressed to one of the virtual network addresses, and wherein the computers communicate operational status information among themselves for each computer on the same subnet using a group membership protocol word to dynamically reconfigure network data traffic assignments among the plurality of computers in response to said status information for said dynamic assignment of virtual network addresses, wherein one computer can configure and monitor the operating condition of another computer in the distributed gateway cluster.

2. A method of controlling data traffic for a computer network through a cluster of a plurality of computers connected as a distributed gateway between a network interface and a subnet over which network data is sent and received, the method comprising:

receiving network data intended for a host on the subnet at said plurality of computers, wherein the network data is addressed to one of a plurality of virtual network addresses selected from a pool of virtual network addresses that are on the subnet and are associated with one or more primary network addresses;

communicating by a computer with one or more of said plurality of computers on the same subnet operational status information about said one or more computers; and

dynamically assigning by said plurality of computers in response to said status information said one virtual network address to a primary network address to send the network data to the intended host on the subnet, wherein a computer can configure and monitor the operating condition of another computer in the distributed gateway cluster.

3. A method of controlling data traffic as defined in claim 2 , wherein said computers comprise server computers that are all members of a first subnet of network addresses over which network data is sent and received, and further comprising:

communicating with the plurality of server computers state information and load information with a dynamic reconfiguration protocol that permits reassignment of network addresses among the server computers in response to said state information and said load information, said status information comprising said state information and said load information; and

communicating between said server computers and a plurality of host computers that are members of a host subnet of network addresses to send and receive network data traffic.

4. A method as defined in claim 3 , wherein communicating with said plurality of server computers comprises sending data using a Reliable Message layer scheme that comprises a token data packet and one or more data carriage packets, wherein the token data packet specifies the number of data carriage packets that together comprise a Reliable Message packet and wherein the data carriage packets include data relating to state information and data traffic load information about each of the server computers.

5. A method as defined in claim 3 , further including:

configuring an operating system of the server computers such that all network addresses in a pool of addresses assigned to the first subnet are assigned to the server computers;

generating a gratuitous address resolution protocol (ARP) message in response to an address reassignment of one server computer and communicating the ARP message to the other server computers of the first subnet;

blocking the sending of an ARP acknowledgment message to the other server computers of the first subnet for any received gratuitous ARP message, thereby inhibiting reboot operation of the respective server computers and ensuring that each server computer is unaware of any duplicate assignment of network address numbers.

6. A method as defined in claim 3 , further including operating as an authoritative node of the first subnet to ensure symmetric routing of network data traffic to and from the first subnet.

7. A method as defined in claim 6 , wherein operating to ensure symmetric traffic routing comprises:

receiving a data request from a responding server computer of the first subnet, wherein the data request was initially received at the responding server computer, which determined the authoritative node for responding to the data request;

identifying a server computer in the first subnet that will handle the data traffic associated with the data request and forwarding the data request to the identified server computer for handling;

receiving a reply message from a server computer of the first subnet that is operating as a default reply node to a second subnet computer that is responding to the data request; and

forwarding the reply message to a server computer of the first subnet that will ensure symmetric routing of the data request and reply message with respect to the server computers of the first subnet.

8. A method as defined in claim 7 , further comprising forwarding assignment information to the server computer of the first subnet that was operating as the default reply node for the data request, wherein the assignment information includes forwarding information that the default reply node can use to directly forward response messages from the second subnet computer to the first subnet computer that will ensure symmetric routing.

9. A data traffic controller as defined in claim 1 , wherein the operational status includes the network data traffic loads being carried by each computer.

10. A data traffic controller as defined in claim 9 , wherein a computer communicates with the other computers on the same subnet to form said distributed gateway.

11. A data traffic controller as defined in claim 10 , wherein the computers dynamically reconfigure the address assignments among the distributed gateway computers in response to traffic loads being carried by the computers.

12. A data traffic controller as defined in claim 11 , wherein a computer determines that it is carrying a traffic load that is less than the traffic load being carried by one of the other computers of said plurality, and in response reconfigures the address assignments to assume a portion of the traffic load being handled by the one other computer and thereby reduces the traffic load of the one other computer.

13. A data traffic controller as defined in claim 11 , wherein the traffic load being carried by a computer of the distributed gateway cluster increases when another one of the computers in the distributed gateway cluster becomes unavailable, and in response the the computers reconfigure the traffic load being handled by the computers.

14. A data traffic controller as defined in claim 11 , wherein operations of the data traffic controller are substantially unchanged after a reconfiguration of the address assignments, thereby making the reconfiguration transparent to a user.

15. A data traffic controller as defined in claim 14 , wherein the operations of the data traffic controller include one or more computer application programs.

16. A data traffic controller as defined in claim 15 , wherein the computer application program comprises a firewall program.

17. A data traffic controller as defined in claim 15 , wherein the computer application program comprises a mail server program.

18. A data traffic controller as defined in claim 15 , wherein the computer application program comprises a file server program.

19. A data traffic controller as defined in claim 10 , wherein upon a new computer communicating with the plurality of computers of the distributed gateway cluster of computers already communicating among themselves, the new computer joins the distributed gateway cluster.

20. A data traffic controller as defined in claim 19 , wherein the computers of the distributed gateway cluster recognize that the new computer has a lower traffic load than the remaining computers of the distributed gateway cluster when the new computer joins the cluster, and in response the new computer assumes a portion of the traffic load from one or more of the computers in a process that dynamically reconfigures the traffic address assignments of the computers.

21. A data traffic controller as defined in claim 19 , wherein the computers of the cluster detect when the new computer begins communicating with the cluster, thereby joining the cluster, and in response the computers perform a dynamic reconfiguration process to change one or more traffic address assignments from the computers to the new computer.

22. A data traffic controller as defined in claim 1 , wherein a computer communicates by passing said group membership protocol word to the other computers along the subnet.

23. A data traffic controller as defined in claim 22 , wherein the group membership protocol word includes node map configuration data indicating traffic assignments for each one of the computers in the distributed gateway cluster.

24. A data traffic controller as defined in claim 22 , wherein the group membership protocol word includes data that indicates the traffic load being carried by each computer in the distributed gateway cluster.

25. A data traffic controller as defined in claim 9 , wherein a computer can receive a user input that specifies a configuration or monitoring change for another computer, and can receive the user input from a remote user.

26. A data traffic controller as defined in claim 10 , wherein the data traffic controller performs the dynamic reconfiguration of address assignments through a graphical user interface (GUI).

27. A data traffic controller as defined in claim 26 , wherein subnet virtual network addresses and traffic address assignments are graphically represented, and the reconfiguration of address assignments can be performed by a drag and drop action of traffic address assignments onto subnet virtual network addresses.

28. A data traffic controller as defined in claim 26 , wherein the GUI includes a local status monitor indicator for the network interface.

29. A data traffic controller as defined in claim 26 , wherein the GUI includes a local condition monitor indicator for the application program of the data traffic controller.

30. A data traffic controller as defined in claim 26 , wherein the GUI includes a local condition monitor indicator for a ping communication utility.

31. A method as defined in claim 2 , wherein the operational status includes the network data traffic loads being carried by each computer.

32. A method as defined in claim 31 , wherein each computer communicates with the other computers on the same subnet and thereby forms a distributed gateway cluster of computers.

33. A method as defined in claim 32 , further including the computers dynamically reconfiguring the address assignments among the distributed gateway cluster computers in response to traffic loads being carried by the computers.

34. A method as defined in claim 33 , wherein reconfiguring comprises a computer determining that it is carrying a traffic load that is less than the traffic load being carried by one of the other computers, and in response reconfiguring the address assignments to assume a portion of the traffic load being handled by another computer and thereby reducing the traffic load of the other computer.

35. A method as defined in claim 33 , wherein the traffic load being carried by the other computer increases when one of the remaining computers in the distributed gateway cluster becomes unavailable, and in response the computers reconfigure the traffic load being handled by the computers.

36. A method as defined in claim 33 , wherein reconfiguring the address assignments results in substantially no change in the operations of the data traffic controller, thereby making the reconfiguration transparent to a user.

37. A method as defined in claim 36 , wherein the operations of the data traffic controller include one or more computer application programs.

38. A method as defined in claim 37 , wherein the computer application program comprises a firewall program.

39. A method as defined in claim 37 , wherein the computer application program comprises a mail server program.

40. A method as defined in claim 37 , wherein the computer application program comprises a file server program.

41. A method as defined in claim 32 , wherein upon a new computer communicating with a computer of the distributed gateway cluster of computers already communicating among themselves, said new computer joins the distributed gateway cluster.

42. A method as defined in claim 41 , wherein the computers of the distributed gateway cluster recognize that a joining computer has a lower traffic load than the computers of the distributed gateway cluster when the computer joins the cluster, and in response the joining computer assumes a portion of the traffic load from one or more of the computers in a process that dynamically reconfigures the traffic address assignments of the cluster.

43. A method as defined in claim 41 , wherein upon detecting when a new computer begins communicating with the cluster, thereby joining the cluster, performing a dynamic reconfiguration process to change one or more traffic address assignments from the computers of the cluster to the new computer.

44. A method as defined in claim 41 , wherein said communicating comprises passing a group membership protocol word to the computers of the subnet.

45. A method as defined in claim 44 , wherein the group membership protocol word includes node map configuration data indicating traffic assignments for each one of the computers in the distributed gateway cluster.

46. A method as defined in claim 44 , wherein the group membership protocol word includes data that indicates the traffic load being carried by each computer in the distributed gateway cluster.

47. A method as defined in claim 31 , wherein a computer can receive a user input that specifies a configuration or monitoring change for another computer, and can receive the user input from a remote user.

48. A method as defined in claim 32 , wherein said dynamically assigning comprises dynamic reconfiguration of address assignments through a graphical user interface (GUI).

49. A method as defined in claim 48 , wherein the subnet virtual network addresses and traffic address assignments are graphically represented, and the reconfiguration of address assignments comprises a drag and drop action of traffic address assignments onto subnet virtual network addresses.

50. A method as defined in claim 48 , wherein the GUI includes a local status monitor indicator for the network interface.

51. A method as defined in claim 48 , wherein the GUI includes a local condition monitor indicator for the application program of the data traffic controller.

52. A method as defined in claim 48 , wherein the GUI includes a local condition monitor indicator for a ping communication utility.

53. A program product for use in a computer of a plurality of computers connected in a cluster as a distributed gateway between a network interface and a subnet of a computer network over which network data is sent and received, wherein the network data is received through the network interface and communicated through the computer cluster to the subnet, the program product comprising:

a recordable medium embodying computer-readable instructions executable by the computers to perform a method comprising:

receiving by said plurality of computers network data intended for a host on the subnet, wherein the network data is addressed to one of a plurality of virtual network addresses selected from a pool of virtual network addresses that are on the subnet and are associated with one or more primary network addresses;

communicating among one or more of said plurality of computers on the same subnet operational status information about said computers; and

dynamically assigning by said plurality of computers of said cluster in response to said communicating said status information said virtual network address to a primary network address to send the network data to the intended host on the subnet, thereby controlling network data traffic, wherein a computer can configure and monitor the operating condition of another computer in the distributed gateway cluster.

54. A program product as defined in claim 53 , wherein the operational status includes the network data traffic loads being carried by each computer.

55. A program product as defined in claim 54 , wherein a computer communicates with the other computers on the same subnet and thereby forms a distributed gateway cluster of computers.

56. A program product as defined in claim 55 , further including dynamically reconfiguring the address assignments among the distributed gateway cluster computers in response to traffic loads being carried by the computers.

57. A program product as defined in claim 56 , wherein said reconfiguring comprises a computer determining that it is carrying a traffic load that is less than the traffic load being carried by one of the other computers, and in response reconfiguring the address assignments to assume a portion of the traffic load being handled by the other computers and thereby reducing the traffic load of the other computers.

58. A program product as defined in claim 56 , wherein the traffic load being carried by the other computer increases when one of the other computers in the distributed gateway cluster becomes unavailable, and in response reconfiguring the traffic load being handled by the other computers.

59. A program product as defined in claim 56 , wherein operations are substantially unchanged after a reconfiguration of the address assignments, thereby making the reconfiguration transparent to a user.

60. A program product as defined in claim 58 , wherein the operations of the data traffic controller include one or more computer application programs.

61. A program product as defined in claim 60 , wherein the computer application program comprises a firewall program.

62. A program product as defined in claim 60 , wherein the computer application program comprises a mail server program.

63. A program product as defined in claim 60 , wherein the computer application program comprises a file server program.

64. A program product as defined in claim 55 , wherein a computer begins communicating with a distributed gateway cluster of computers already communicating among themselves, thereby joining the distributed gateway cluster.

65. A program product as defined in claim 64 , wherein the computers of the distributed gateway cluster recognize that a computer has a lower traffic load than the remaining computers of the distributed gateway cluster when the computer joins the cluster, and in response the computer assumes a portion of the traffic load from one or more of the computers in a process that dynamically reconfigures the traffic address assignments of the cluster.

66. A program product as defined in claim 64 , wherein the computers detect when a new computer begins communicating with the cluster, thereby joining the cluster, and in response the computers perform a dynamic reconfiguration process to change one or more traffic address assignments to the new computer.

67. A program product as defined in claim 54 , wherein a computer communicates by passing a group membership protocol word to the other computers along the subnet.

68. A program product as defined in claim 67 , wherein the group membership protocol word includes node map configuration data indicating traffic assignments for each one of the computers in the distributed gateway cluster.

69. A program product as defined in claim 67 , wherein the group membership protocol word includes data that indicates the traffic load being carried by each computer in the distributed gateway cluster.

70. A program product as defined in claim 54 , wherein a computer can receive a user input that specifies a configuration or monitoring change for another computer, and can receive the user input from a remote user.

71. A program product as defined in claim 55 , wherein the data traffic controller performs the dynamic reconfiguration of address assignments through a graphical user interface (GUI).

72. A program product as defined in claim 71 , wherein the subnet virtual network addresses and traffic address assignments are graphically represented, and the reconfiguration of address assignments can be performed by a drag and drop action of traffic address assignments onto subnet virtual network addresses.

73. A program product as defined in claim 71 , wherein the GUI includes a local status monitor indicator for the network interface.

74. A program product as defined in claim 71 , wherein the GUI includes a local condition monitor indicator for the application program of the data traffic controller.

75. A program product as defined in claim 71 , wherein the GUI includes a local condition monitor indicator for a ping communication utility.

Assignments (9)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: EMC CORPORATION
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 040203/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 21, 2006
From: RAINFINITY
To: EMC CORPORATION
Reel/Frame 017596/0063 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2000
From: BRUCK, JEHOSHUA; BOHOSSIAN, VASKEN; FAN, CHENGGONG; LEMAHIEU, PAUL; LOVE, PHILIP
To: RAINFINITY, INC.
Reel/Frame 011291/0880 →
Continuity (1)
Continuation 0943763700 · Nov 10, 1999