Building a mesh virtual private network (VPN) in a hybrid or multi-cloud cluster

1. A system for building a mesh virtual private network (VPN) in a cloud cluster, the system comprising:

the cloud cluster including a first set of network nodes in a private cloud connected with a second set of nodes of a public cloud;

each of the first and second nodes having a respective operating system including a VPN node discovery controller having:

a VPN node configuration data controller configured to discover nodes of the VPN by determining IP addresses and port addresses of the first and second set of nodes of the cloud cluster from discovery agents including:

an internal discovery agent of each of the first and second nodes that includes a Kubernetes API and that uses an annotation-marked Kubernetes node resource to determine IP addresses and port addresses of some of the first and second nodes;

a node discovery agent located within a node of the first and second nodes that includes a Kubernetes API and that determine IP addresses and port addresses of other nodes of the first and second nodes; and

an external discovery agent located outside the cloud cluster that includes a Kubernetes API and that determines IP addresses and port addresses of a least of the first and second nodes;

a VPN configuration data generator configured to generate VPN configuration data for at least one VPN node of the first set of nodes and at least one VPN node of the second set of nodes using the determined IP addresses and port addresses; and

a VPN builder configured to build a configuration of the VPN of the hybrid or multi-cloud cluster for the VPN nodes using the VPN configuration data.

2. The system of claim 1 , wherein VPN node configuration data controller includes a private address identifier to discover nodes of the VPN by:

identifying private single network address translated IP addresses and private port addresses of the first and second set of nodes from public ones of the determined IP addresses and port addresses of the first and second set of the VPN nodes by automatically penetrating network address translation (NAT);

identifying double network address translated IP addresses and private port addresses of the first and second set of nodes from public ones of the determined IP addresses and port addresses of the first and second set of the VPN nodes by one of automatically penetrating NAT or using additional tooling external to the operating system.

3. The system of claim 1 , wherein discovering nodes of the VPN includes determining multiple endpoint IP addresses and port addresses; shared cryptographic public key; and a list of accepted IP address for that node to connect to, for each of the first and second set of nodes of the cloud cluster from the discovery agents.

4. The system of claim 1 , wherein discovering nodes of the VPN includes

each discovery agent determining the IP addresses and port addresses of some of the first and second set of nodes of the cloud cluster from cluster node configuration data of the clusters that is received by the discovery agents, and sending the determined IP addresses and port addresses of some of the first and second set of nodes to the VPN node configuration data controller; and

the VPN node configuration data controller combining the determined IP addresses and port addresses of some of the first and second set of nodes.

5. The system of claim 1 , wherein the cluster includes nodes that periodically provide the external discovery agent with VPN node IP addresses and port addresses, and

wherein discovering nodes further comprises the external discovery agent periodically updating the first and second nodes operating systems with the determined IP addresses and port addresses.

6. The system of claim 1 , wherein the external discovery agent:

is a server that is not one of the first or second nodes;

collates the IP addresses and port addresses;

sends the collated IP addresses and port addresses to the first and second nodes.

7. The system of claim 1 , wherein

the VPN allows data and applications to be shared between the public cloud and private cloud;

each operating system is configured to automatically begin and periodically repeat determining, generating and building upon user activation of an enablement flag of the operating system;

building includes deterministically generating the configuration of the VPN from a default configuration; and

building the configuration of the VPN includes:

each of the nodes of the first and second set of nodes 1) connecting with most of the other nodes of the first and second set of nodes based on the VPN configuration data; and 2) sharing registry data with the connected to other nodes of the first and second set of nodes.

8. The system of claim 1 , each operating system further comprising an addressing controller configured to create a globally-unique local address for each IP address of each VPN node by:

using a reserved address space range for the VPN nodes; and

deterministically selecting for each VPN node, a prefix and an address range of the reserved address space range to create the globally-unique local address for each IP address of each VPN node.

9. The system of claim 1 , each operating system further comprising an integral controller configured to:

keep a list of expected peer nodes of the VPN nodes up-to-date by merging all data from each of discovery agent that IP addresses and port addresses are being received from, keyed by the public key of each expected peer;

for each of expected peer nodes of the VPN nodes that communication is not received from for at least a predetermined period of time minutes, cycle through available potential IP addresses for the peer from the discovery agents, until such time as the integral controller forms a connection with the expected peer node of the VPN nodes that communication is not received from; and

maintain the connection until lost.

10. The system of claim 1 , each operating system further comprising a routing engine configured to:

mark data packets sent between the VPN nodes with a packet interface source;

discover data packets sent between the VPN nodes that are not marked as being sent from a packet interface source of one of said operating systems;

mark packets which are not marked as being sent from a packet interface source of one of said operating systems to be sent to a packet interface source of one of said operating systems, wherein marking the packets which are not marked works in tandem with IP rules in the routing engine to pass all packets which are not marked through a packet interface source one of said operating systems; and

maintain a full set of cluster targets of the VPN nodes for that operating system under a table along with rules to mark the packets to be sent to a packet interface source of one of said operating systems.

11. The system of claim 1 , wherein the node discover agent is a network route node configured to communicate data packets between nodes of the VPN nodes that are not directly connected:

wherein the node discover agent is dynamically chosen as one of the first and second network nodes for a given Layer 3 subnet of a plurality of Layer 3 subnets of the first and second network nodes,

wherein the node discover agent has connectivity between all known Layer 3 subnets of the given Layer 3 subnet of the first and second network nodes.

12. A method for building a mesh virtual private network (VPN) in a cloud cluster including a first set of network nodes in a private cloud connected with a second set of nodes of a public cloud, the method comprising:

each of the first and second nodes:

discovering nodes of the VPN by determining total VPN node configuration data of the first and second set of nodes of the cloud cluster from portions of the VPN node configuration data received from discovery agents including:

an internal discovery agent of each of the first and second nodes that includes a Kubernetes API, and that is for determining a portion of VPN node configuration data of some of the first and second nodes;

a node discovery agent located within a node of the first and second nodes that includes a Kubernetes API, and that is for determining a portion of VPN node configuration data of other nodes of the first and second nodes; and

an external discovery agent located outside the cloud cluster that includes a Kubernetes API, and that is for determining a portion of VPN node configuration data of at least some nodes of the first and second nodes;

generating VPN configuration data for at least one VPN node of the first set of nodes and at least one VPN node of the second set of nodes using the total VPN node configuration data; and

building a configuration of the VPN of the hybrid or multi-cloud cluster for the VPN nodes using the generated VPN configuration data.

13. The method of claim 12 , wherein determining total VPN node configuration data includes:

1) identifying private single network address translated IP addresses and private port addresses of the first and second set of nodes from public ones of the determined total VPN node configuration data of the first and second set of the VPN nodes by automatically penetrating network address translation (NAT);

2) identifying double network address translated IP addresses and private port addresses of the first and second set of nodes from public ones of the determined total VPN node configuration data of the first and second set of the VPN nodes by one of automatically penetrating NAT or using additional tooling external to the operating system.

14. The method of claim 12 , wherein determining total VPN node configuration data includes:

determining multiple endpoint IP addresses and port addresses; shared cryptographic public key; and a list of accepted IP address for that node to connect to, for each of the first and second set of nodes of the cloud cluster from the portions of VPN node configuration data from the discovery agents.

15. The method of claim 12 , wherein discovering nodes of the VPN includes:

each discovery agent determining the portion of VPN node configuration data of some of the first and second set of nodes of the cloud cluster from cluster node configuration data of the clusters that is received by the discovery agents, and sending the determined portion of VPN node configuration data of some of the first and second set of nodes to the VPN node configuration data controller; and

the VPN node configuration data controller combining the determined portions of VPN node configuration data of some of the first and second set of nodes into the total VPN node configuration data.

16. The method of claim 12 , wherein discovering nodes of the VPN includes:

the external discovery agent periodically updating the first and second nodes operating systems with the determined portion of VPN node configuration data.

17. The method of claim 12 , wherein the external discovery agent determining a portion of VPN node configuration data includes:

collating the portion of VPN node configuration data;

sending the collated portion of VPN node configuration data to the first and second nodes.

18. The method of claim 12 , wherein each operating system is configured to automatically begin and periodically repeat determining, generating and building upon user activation of an enablement flag of the operating system;

wherein building includes deterministically generating the configuration of the VPN from a default configuration; and

wherein building the configuration of the VPN includes each of the nodes of the first and second set of nodes 1) connecting with most of the other nodes of the first and second set of nodes based on the configuration of the VPN; and 2) sharing registry data with the connected to other nodes of the first and second set of nodes.

19. The method of claim 12 , each of the first and second nodes further creating a globally-unique local address for each IP address of each VPN node by:

using a reserved IPv6 address space range for the VPN nodes; and

deterministically selecting for each VPN node, a prefix and an address range of the reserved IPv6 address space range to create the globally-unique local address for each IP address of each VPN node.

20. The method of claim 12 , each of the first and second nodes further:

keeping a list of expected peer nodes of the VPN nodes up-to-date by merging all data from each of discovery agent that the portion of VPN node configuration data are being received from, keyed by the public key of each expected peer;

for each of expected peer nodes of the VPN nodes that communication is not received from for a predetermined period of times, cycling through available potential IP addresses for the peer from the discovery agents, until such time as the integral controller forms a connection with the expected peer node of the VPN nodes that communication is not received from; and

maintaining the connection until lost.

21. The method of claim 12 , each of the first and second nodes further:

marking data packets sent between the VPN nodes with a packet interface source;

discovering data packets sent between the VPN nodes that are not marked as being sent from a packet interface source of one of said operating systems;

marking packets which are not marked as being sent from a packet interface source of one of said operating systems to be sent to a packet interface source of one of said operating systems, wherein marking the packets which are not marked works in tandem with IP rules in the routing engine to pass all packets which are not marked through a packet interface source one of said operating systems; and

maintaining a full set of cluster targets of the VPN nodes for that operating system under a table along with rules to mark the packets to be sent to a packet interface source of one of said operating systems.

22. The method of claim 12 , further comprising:

one of the first and second nodes further dynamically choosing the node discover agent as one of the first and second network nodes for a given Layer 3 subnet of a plurality of Layer 3 subnets of the first and second network nodes, wherein the node discover agent has connectivity between all known Layer 3 subnets of the given Layer 3 subnet of the first and second network nodes; and

the node discover agent communicating data packets between nodes of the VPN nodes that are not directly connected.

23. A non-volatile machine readable medium storing a program having instructions which when executed by a processor will cause the processor to build a mesh virtual private network (VPN) in a cloud cluster including a private cloud having a first set of network nodes connected to a second set of nodes of a public cloud, the instructions of the program for:

discovering nodes of the VPN by determining total VPN node configuration data of the first and second set of nodes of the cloud cluster from portions of the VPN node configuration data received from discovery agents including:

an internal discovery agent of each of the first and second nodes that includes a Kubernetes API, and that is for determining a portion of VPN node configuration data of some of the first and second nodes;

a node discovery agent located within a node of the first and second nodes includes a Kubernetes API, and that is for determining a portion of VPN node configuration data of other nodes of the first and second nodes; and

an external discovery agent located outside the Kubernetes cluster that includes a Kubernetes API, and that is for determining a portion of VPN node configuration data of at least some nodes of the first and second nodes;

generating VPN configuration data for at least one VPN node of the first set of nodes and at least one VPN node of the second set of nodes using the total VPN node configuration data; and

building a configuration of the VPN of the hybrid or multi-cloud cluster for the VPN nodes using the generated VPN configuration data.

24. The medium of claim 23 , wherein determining total VPN node configuration data includes:

identifying private single network address translated IP addresses and private port addresses of the first and second set of nodes from public ones of the determined total VPN node configuration data of the first and second set of the VPN nodes by automatically penetrating network address translation (NAT);

identifying double network address translated IP addresses and private port addresses of the first and second set of nodes from public ones of the determined total VPN node configuration data of the first and second set of the VPN nodes by one of automatically penetrating NAT or using additional tooling external to the operating system.

25. The medium of claim 23 , wherein determining total VPN node configuration data includes determining multiple endpoint IP addresses and port addresses; shared cryptographic public key; and a list of accepted IP address for that node to connect to, for each of the first and second set of nodes of the cloud cluster from the portions of VPN node configuration data from the discovery agents.

26. The medium of claim 23 , wherein discovering nodes of the VPN includes:

each discovery agent determining the portion of VPN node configuration data of some of the first and second set of nodes of the cloud cluster from cluster node configuration data of the clusters that is received by the discovery agents, and sending the determined portion of VPN node configuration data of some of the first and second set of nodes to the VPN node configuration data controller; and

the VPN node configuration data controller combining the determined portions of VPN node configuration data of some of the first and second set of nodes into the total VPN node configuration data.