Multi-cluster networking using hub and spoke elastic mesh
A method is provided that includes receiving, from a first cluster, data intended for a second cluster and routing the data to the second cluster, thereby forming an indirect connection between a first cluster and the second cluster. A network usage of the indirect connection may be monitored and in response to determining that the network usage of the indirect connection exceeds a network usage threshold, a respective remote agent of each of the first and second clusters may be instructed to form a direct connection between the first and second clusters and remove the indirect connection.
1. A method comprising:
receiving from a first cluster, data intended for a second cluster;
routing the data to the second cluster to form an indirect connection between the first cluster and the second cluster, wherein the indirect connection comprises a hub between the first and second clusters;
monitoring a network usage of the indirect connection; and
in response to determining that the network usage of the indirect connection exceeds a network usage threshold indicative of a maximum latency of the indirect connection:
instructing a respective remote agent executing on each of the first and second clusters to create a direct tunnel connection between the first and second clusters;
removing the indirect connection between the first and second clusters by updating routing tables stored on the first and second clusters; and
storing, on the hub, a record indicating the removal of the indirect connection and the creation of the direct tunnel connection.
2. The method of claim 1 , further comprising querying the respective remote agent of each of the first and second clusters to determine a network usage information of the direct tunnel connection.
3. The method of claim 1 , further comprising:
recording the direct tunnel connection in a connection table;
in response to determining that the network usage of the direct tunnel connection is below the network usage threshold, instructing the respective remote agent of each of the first and second clusters to remove the direct tunnel connection; and
removing the direct tunnel connection record from the connection table.
4. The method of claim 1 , wherein the direct tunnel connection is a virtual private network (VPN) tunnel.
5. The method of claim 1 , wherein the respective remote agent of each of the first and second clusters advertise a cluster address and an internet protocol (IP) address of one or more of nodes in their respective cluster to the hub.
6. The method of claim 1 , wherein the direct tunnel connection does not comprise the hub between the first and second clusters.
7. A system comprising:
a memory to store one or more network usage thresholds; and
a processing device, operatively coupled to the memory, the processing device to:
receive from a first cluster, data intended for a second cluster;
route the data to the second cluster to form an indirect connection between the first cluster and the second cluster, wherein the indirect connection comprises a hub between the first and second clusters;
monitor a network usage of the indirect connection; and
in response to determining that the network usage of the indirect connection exceeds a network usage threshold indicative of a maximum latency of the indirect connection:
instruct a respective remote agent executing on each of the first and second clusters to create a direct tunnel connection between the first and second clusters;
remove the indirect connection between the first and second clusters by updating routing tables stored on the first and second clusters; and
store, on the hub, a record indicating the removal of the indirect connection and the creation of the direct tunnel connection.
8. The system of claim 7 , wherein the processing device further to query the respective remote agent of each of the first and second clusters to determine a network usage information of the direct tunnel connection.
9. The system of claim 7 , wherein the processing device is further to:
record the direct tunnel connection is a connection table;
in response to determining that the network usage of the direct tunnel connection is below the network usage threshold, instruct the respective remote agent of each of the first and second clusters to remove the direct tunnel connection; and
remove the direct tunnel connection record from the connection table.
10. The system of claim 7 , wherein the direct tunnel connection is a virtual private network (VPN) tunnel.
11. The system of claim 7 , wherein the respective remote agent of each of the first and second clusters advertise a cluster address and an internet protocol (IP) address of one or more of nodes in their respective cluster to the hub.
12. The system of claim 7 , wherein the direct tunnel connection does not comprise the hub between the first and second clusters.
13. A non-transitory computer readable storage medium, having instructions stored thereon that, when executed by a processing device, cause the processing device to:
receive from a first cluster, data intended for a second cluster;
route the data to the second cluster to form an indirect connection between the first cluster and the second cluster, wherein the indirect connection comprises a hub between the first and second clusters;
monitor a network usage of the indirect connection; and
in response to determining that the network usage of the indirect connection exceeds a network usage threshold indicative of a maximum latency of the indirect connection:
instruct, by the processing device, a respective remote agent executing on each of the first and second clusters to create a direct tunnel connection between the first and second clusters;
remove the indirect connection between the first and second clusters by updating routing tables stored on the first and second clusters; and
store, on the hub, a record indicating the removal of the indirect connection and the creation of the direct tunnel connection.
14. The non-transitory computer readable storage medium of claim 13 , wherein the processing device further to query the respective remote agent of each of the first and second clusters to determine a network usage information of the direct tunnel connection.
15. The non-transitory computer readable storage medium of claim 13 , wherein the processing device is further to:
record the direct tunnel connection is a connection table;
in response to determining that the network usage of the direct tunnel connection is below the network usage threshold, instruct the respective remote agent of each of the first and second clusters to remove the direct tunnel connection; and
remove the direct tunnel connection record from the connection table.
16. The non-transitory computer readable storage medium of claim 15 , wherein the direct tunnel connection is a virtual private network (VPN) tunnel.
17. The non-transitory computer readable storage medium of claim 15 , wherein the respective remote agent of each of the first and second clusters advertise a cluster address and an internet protocol (IP) address of one or more of nodes in their respective cluster to the hub.