Cloud-based virtualized data storage system with tunnel-based inter-node communications

1 . A cloud-based virtualized data storage system comprising:

a cloud computing infrastructure having compute nodes interconnected by a cloud network and executing co-operating instances of virtual storage appliance software to realize respective virtual storage appliances (VSAs),

wherein the VSAs have respective virtualized network interfaces to the cloud network and to cloud storage providing underlying storage for logical storage provided to data storage clients by the VSAs,

and wherein each VSA has a respective pair of storage processor (SP) nodes each having (1) an inter-node (IN) network interface for communicating with the other SP node of the same VSA and an intra-cluster (IC) network interface for communicating with other VSAs of a respective cluster, the IN and IC network interfaces using private network addresses not routable in the cloud network, and (2) a network virtualization layer connecting the IN and IC network interfaces to an IN tunnel and an IC tunnel, respectively,

the tunnels being carried in the cloud network using corresponding cloud network addresses routable in the cloud network, the IN tunnel providing inter-node communications between the SP nodes of a VSA, the IC tunnel providing inter-VSA communications among VSAs of a cluster,

wherein each VSA is realized as a collection of virtual machines each executing an image of a data storage processing node, and

wherein the cloud network addresses are auto-allocated cloud native underlay Internet Protocol (IP) addresses of respective ports of the virtual machines, and wherein the VSAs perform control-plane functions that include upon deployment, querying the auto-allocated cloud native underlay IP addresses of the ports of the virtual machines and setting up both the IN and IC tunnels using the underlay IP addresses.

2 . The cloud-based virtualized data storage system of claim 1 , wherein the compute nodes are virtual-computing host computers that execute applications using virtual machines and a supervisory mechanism.

3 . The cloud-based virtualized data storage system of claim 2 , wherein the control-plane functions performed by the VSAs further include continually monitoring the ongoing use of the underlay IP addresses by the virtual machines, and, in the event that an underlay IP address is changed, re-orchestrating the tunnels on all affected storage processing nodes without affecting operation of higher-level applications and services due to their use of private IN and IC networks carried by the tunnels.

4 . The cloud-based virtualized data storage system of claim 3 , wherein the underlay IP address is changed due to one or more of an error, a reconfiguration, or a recovery.

5 . The cloud-based virtualized data storage system of claim 1 , wherein the VSAs collectively provide a variety of data storage services including block and file storage along with related data storage services including one or more of replication, snapshots, and de-duplication.

6 . The cloud-based virtualized data storage system of claim 1 , wherein each storage processing node executes a software image used to realize a counterpart storage processing node in a separate purpose-built data storage system environment, including a network configuration using the private network addresses which are routable in the purpose-built data storage system environment.

7 . The cloud-based virtualized data storage system of claim 1 , wherein the network virtualization layers use transparent encryption of traffic flowing via the tunnels so that data packets flowing via the tunnels are encrypted.

8 . The cloud-based virtualized data storage system of claim 1 , wherein the private network addresses of the IN and IC network interfaces are hardcoded.

9 . The cloud-based virtualized data storage system of claim 1 , wherein the private network addresses of the IN and IC network interfaces are auto-generated.

10 . The cloud-based virtualized data storage system of claim 9 , wherein the private network addresses of the IN and IC network interfaces are auto generated from an IPv6 unique local address (ULA) range.

11 . A method of providing data storage services using a cloud-based virtualized data storage system, comprising:

instantiating and operating a plurality of instances virtual storage appliance software to realize respective virtual storage appliances (VSAs), wherein the VSAs have respective virtualized network interfaces to a cloud network and to cloud storage providing underlying storage for logical storage provided to data storage clients by the VSAs, and wherein each VSA has a respective pair of storage processor (SP) nodes each having (1) an inter-node (IN) network interface for communicating with the other SP node of the same VSA and an intra-cluster (IC) network interface for communicating with other VSAs of a respective cluster, the IN and IC network interfaces using private network addresses not routable in the cloud network, and (2) a network virtualization layer connecting the IN and IC network interfaces to an IN tunnel and an IC tunnel, respectively;

instantiating and operating the IN and IC tunnels in the cloud network using corresponding cloud network addresses routable in the cloud network, the IN tunnel operated to provide inter-node communications between the SP nodes of a VSA, the IC tunnel operated to provide inter-VSA communications among VSAs of a cluster;

wherein each VSA is realized as a collection of virtual machines each executing an image of a data storage processing node; and

wherein the cloud network addresses are auto-allocated cloud native underlay Internet Protocol (IP) addresses of respective ports of the virtual machines, and wherein the VSAs perform control-plane functions that include upon deployment, querying the auto-allocated cloud native underlay IP addresses of the ports of the virtual machines and setting up both the IN and IC tunnels using the underlay IP addresses.

12 . The method of claim 11 , wherein the control-plane functions performed by the VSAs further include continually monitoring the ongoing use of the underlay IP addresses by the virtual machines, and, in the event that an underlay IP address is changed, re-orchestrating the tunnels on all affected storage processing nodes without affecting operation of higher-level applications and services due to their use of private IN and IC networks carried by the tunnels.

13 . The method of claim 12 , wherein the underlay IP address is changed due to one or more of an error, a reconfiguration, or a recovery.

14 . The method of claim 11 , wherein the VSAs collectively provide a variety of data storage services including block and file storage along with related data storage services including one or more of replication, snapshots, and de-duplication.

15 . The method of claim 11 , wherein each storage processing node executes a software image used to realize a counterpart storage processing node in a separate purpose-built data storage system environment, including a network configuration using the private network addresses which are routable in the purpose-built data storage system environment.

16 . The method of claim 11 , wherein the network virtualization layers use transparent encryption of traffic flowing via the tunnels so that data packets flowing via the tunnels are encrypted.

17 . The method of claim 11 , wherein the private network addresses of the IN and IC network interfaces are hardcoded.

18 . The method of claim 11 , wherein the private network addresses of the IN and IC network interfaces are auto-generated.

19 . The method of claim 18 , wherein the private network addresses of the IN and IC network interfaces are auto generated from an IPv6 unique local address (ULA) range.