HIBERNATING AND RESUMING NODES OF A COMPUTING CLUSTER
Methods, systems and computer program products for hibernating a computing cluster. The present disclosure describes techniques for hibernating and resuming nodes of a computing cluster and entire computing clusters including movement of data and metadata to and from a cloud-tier storage facility (e.g., a cloud disk(s)) in an efficient manner.
1 . A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes a set of acts, the set of acts comprising:
maintaining a cluster on a plurality of nodes of a virtualization system, the cluster having a distributed metadata system that uses a persistent data structure to store system metadata, wherein a storage pool of the cluster is managed using the system metadata, the system metadata is stored on multiple nodes of the plurality of nodes, and a first node of the multiple nodes stores a primary copy of a first portion of the system metadata and a redundant copy of a second portion of the system metadata;
determining that the system metadata is to be migrated to a backup storage that is external to the cluster; and
migrating system metadata to the backup storage at least by transmitting the primary copy of the first portion of the system metadata from the first node to the backup storage without transmitting the redundant copy of the second portion of the system metadata from first node to the backup storage.
2 . The computer readable medium of claim 1 , wherein the multiple nodes transfer different portions of the system metadata to the backup storage, and the different portions comprise different entries in the persistent data structure.
3 . The computer readable medium of claim 1 , wherein the primary copies of portions of the system metadata comprise different entries in the persistent data structure that are transferred by the multiple nodes using multiple connections on each of the multiple nodes.
4 . The computer readable medium of claim 1 , wherein the persistent data structure comprises a sorted string table (SST) and the SST is divided into multiple ranges of SST entries using a column family and token range.
5 . The computer readable medium of claim 4 , wherein the multiple ranges of SST entries are further divided into multiple subsets of SST entries by the multiple nodes using the column family and token ranges.
6 . The computer readable medium of claim 5 , wherein the set of acts further comprise, prior to migration, compacting the multiple subsets of SST entries, generating snapshots of respective ones of the multiple subsets of SST entries, or validating the multiple subsets of SST entries.
7 . The computer readable medium of claim 1 , wherein the backup storage comprises a cloud disk and the storage pool is constructed from a plurality of storage devices directly attached to respective nodes of the plurality of nodes.
8 . The computer readable medium of claim 1 , wherein the set of acts further comprise:
identifying a plurality of data items having a corresponding redundant copy on the storage pool;
selecting a single copy of each of the plurality of data items for transfer to the backup storage; and
migrating the single copy of each of the plurality of data items to the backup storage without copying multiple copies of an individual data item to the backup storage.
9 . A system comprising:
a storage medium having stored thereon a sequence of instructions; and
a processor that executes the sequence of instructions to cause the processor to perform a set of acts comprising:
maintaining a cluster on a plurality of nodes of a virtualization system, the cluster having a distributed metadata system that uses a persistent data structure to store system metadata, wherein a storage pool of the cluster is managed using the system metadata, the system metadata is stored on multiple nodes of the plurality of nodes, and a first node of the multiple nodes stores a primary copy of a first portion of the system metadata and a redundant copy of a second portion of the system metadata;
determining that the system metadata is to be migrated to a backup storage that is external to the cluster; and
migrating system metadata to the backup storage at least by transmitting the primary copy of the first portion of the system metadata from the first node to the backup storage without transmitting the redundant copy of the second portion of the system metadata from first node to the backup storage.
10 . The system of claim 9 , wherein the multiple nodes transfer different portions of the system metadata to the backup storage, and the different portions comprise different entries in the persistent data structure.
11 . The system of claim 9 , wherein the primary copies of portions of the system metadata comprise different entries in the persistent data structure that are transferred by the multiple nodes using multiple connections on each of the multiple nodes.
12 . The system of claim 9 , wherein the persistent data structure comprises a sorted string table (SST) and the SST is divided into multiple ranges of SST entries using a column family and token range.
13 . The system of claim 12 , wherein the multiple ranges of SST entries are further divided into multiple subsets of SST entries by the multiple nodes using the column family and token ranges.
14 . The system of claim 13 , wherein the set of acts further comprise, prior to migration, compacting the multiple subsets of SST entries, generating snapshots of respective ones of the multiple subsets of SST entries, or validating the multiple subsets of SST entries.
15 . The system of claim 9 , wherein the backup storage comprises a cloud disk and the storage pool is constructed from a plurality of storage devices directly attached to respective nodes of the plurality of nodes.
16 . The system of claim 9 , wherein the set of acts further comprise:
identifying a plurality of data items having a corresponding redundant copy on the storage pool;
selecting a single copy of each of the plurality of data items for transfer to the backup storage; and
migrating the single copy of each of the plurality of data items to the backup storage without copying multiple copies of an individual data item to the backup storage.
17 . A method comprising:
maintaining a cluster on a plurality of nodes of a virtualization system, the cluster having a distributed metadata system that uses a persistent data structure to store system metadata, wherein a storage pool of the cluster is managed using the system metadata, the system metadata is stored on multiple nodes of the plurality of nodes, and a first node of the multiple nodes stores a primary copy of a first portion of the system metadata and a redundant copy of a second portion of the system metadata;
determining that the system metadata is to be migrated to a backup storage that is external to the cluster; and
migrating system metadata to the backup storage at least by transmitting the primary copy of the first portion of the system metadata from the first node to the backup storage without transmitting the redundant copy of the second portion of the system metadata from first node to the backup storage.
18 . The method of claim 17 , wherein the multiple nodes transfer different portions of the system metadata to the backup storage, and the different portions comprise different entries in the persistent data structure.
19 . The method of claim 17 , wherein the primary copies of portions of the system metadata comprise different entries in the persistent data structure that are transferred by the multiple nodes using multiple connections on each of the multiple nodes.
20 . The method of claim 17 , wherein the persistent data structure comprises a sorted string table (SST) and the SST is divided into multiple ranges of SST entries using a column family and token range.
21 . The method of claim 20 , wherein the multiple ranges of SST entries are further divided into multiple subsets of SST entries by the multiple nodes using the column family and token ranges.
22 . The method of claim 21 , further comprises, prior to migration, compacting the multiple subsets of SST entries, generating snapshots of respective ones of the multiple subsets of SST entries, or validating the multiple subsets of SST entries.
23 . The method of claim 17 , wherein the backup storage comprises a cloud disk and the storage pool is constructed from a plurality of storage devices directly attached to respective nodes of the plurality of nodes.
24 . The method of claim 17 , further comprises:
identifying a plurality of data items having a corresponding redundant copy on the storage pool;
selecting a single copy of each of the plurality of data items for transfer to the backup storage; and
migrating the single copy of each of the plurality of data items to the backup storage without copying multiple copies of an individual data item to the backup storage.