Hibernating and resuming nodes of a computing cluster
Methods, systems and computer program products for hibernating a computing cluster. The present disclosure describes techniques for hibernating and resuming nodes of a computing cluster and entire computing clusters including movement of data and metadata to and from a cloud-tier storage facility (e.g., a cloud disk(s)) in an efficient manner.
1 . A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, cause a set of acts, the set of acts comprising:
maintaining a cluster on a plurality of nodes of a virtualization system, the cluster having a distributed metadata system that uses a persistent data structure to store system metadata, wherein a storage pool of the cluster is managed using the system metadata, the system metadata is stored on multiple nodes of the plurality of nodes, the system metadata comprises a plurality of portions of metadata, individual portions of the plurality of portions of metadata are stored as both a primary copy on a node of the multiple nodes and at least one redundant copy on at least one other node of the multiple nodes, and at least two nodes of the multiple nodes store both a primary copy of at least one individual portion and a redundant copy of at least one other individual portion;
determining that the system metadata is to be migrated to a backup storage that is external to the cluster; and
migrating the system metadata to the backup storage at least by:
identifying and selecting, from the primary and redundant copies, a single copy of the individual portions of the plurality of portions of metadata for transfer to the backup storage, wherein the primary copy of the individual portions of the plurality of portions of metadata is selected as the single copy of the individual portions of the plurality of portions of metadata, and
transmitting the identified and selected single copy of the individual portions of the plurality of portions of metadata from the multiple nodes to the backup storage.
2 . The computer readable medium of claim 1 , wherein the multiple nodes transfer different portions of the system metadata to the backup storage, and the different portions comprise different entries in the persistent data structure.
3 . The computer readable medium of claim 1 , wherein the primary copies of portions of the system metadata comprise different entries in the persistent data structure that are transferred by the multiple nodes using multiple connections on each of the multiple nodes.
4 . The computer readable medium of claim 1 , wherein the persistent data structure comprises a sorted string table (SST) and the SST is divided into multiple ranges of SST entries using a column family and token range.
5 . The computer readable medium of claim 4 , wherein the multiple ranges of SST entries are further divided into multiple subsets of SST entries by the multiple nodes using the column family and token ranges.
6 . The computer readable medium of claim 5 , wherein the set of acts further comprise, prior to migration, compacting the multiple subsets of SST entries, generating snapshots of respective ones of the multiple subsets of SST entries, or validating the multiple subsets of SST entries.
7 . The computer readable medium of claim 1 , wherein the backup storage comprises a cloud disk and the storage pool is constructed from a plurality of storage devices directly attached to respective nodes of the plurality of nodes.
8 . The computer readable medium of claim 1 , wherein the set of acts further comprise:
identifying a plurality of data items having a corresponding redundant copy on the storage pool;
selecting a single copy of each of the plurality of data items for transfer to the backup storage; and
migrating the single copy of each of the plurality of data items to the backup storage without copying multiple copies of an individual data item to the backup storage.
9 . A system comprising:
a storage medium having stored thereon a sequence of instructions; and
a processor that executes the sequence of instructions to cause the processor to perform a set of acts comprising:
maintaining a cluster on a plurality of nodes of a virtualization system, the cluster having a distributed metadata system that uses a persistent data structure to store system metadata, wherein a storage pool of the cluster is managed using the system metadata, the system metadata is stored on multiple nodes of the plurality of nodes, the system metadata comprises a plurality of portions of metadata, individual portions of the plurality of portions of metadata are stored as both a primary copy on a node of the multiple nodes and at least one redundant copy on at least one other node of the multiple nodes, and at least two nodes of the multiple nodes store both a primary copy of at least one individual portion and a redundant copy of at least one other individual portion;
determining that the system metadata is to be migrated to a backup storage that is external to the cluster; and
migrating the system metadata to the backup storage at least by:
identifying and selecting, from the primary and redundant copies, a single copy of the individual portions of the plurality of portions of metadata for transfer to the backup storage, wherein the primary copy of the individual portions of the plurality of portions of metadata is selected as the single copy of the individual portions of the plurality of portions of metadata, and
transmitting the identified and selected single copy of the individual portions of the plurality of portions of metadata from the multiple nodes to the backup storage.
10 . The system of claim 9 , wherein the multiple nodes transfer different portions of the system metadata to the backup storage, and the different portions comprise different entries in the persistent data structure.
11 . The system of claim 9 , wherein the primary copies of portions of the system metadata comprise different entries in the persistent data structure that are transferred by the multiple nodes using multiple connections on each of the multiple nodes.
12 . The system of claim 9 , wherein the persistent data structure comprises a sorted string table (SST) and the SST is divided into multiple ranges of SST entries using a column family and token range.
13 . The system of claim 12 , wherein the multiple ranges of SST entries are further divided into multiple subsets of SST entries by the multiple nodes using the column family and token ranges.
14 . The system of claim 13 , wherein the set of acts further comprise, prior to migration, compacting the multiple subsets of SST entries, generating snapshots of respective ones of the multiple subsets of SST entries, or validating the multiple subsets of SST entries.
15 . The system of claim 9 , wherein the backup storage comprises a cloud disk and the storage pool is constructed from a plurality of storage devices directly attached to respective nodes of the plurality of nodes.
16 . The system of claim 9 , wherein the set of acts further comprise:
identifying a plurality of data items having a corresponding redundant copy on the storage pool;
selecting a single copy of each of the plurality of data items for transfer to the backup storage; and
migrating the single copy of each of the plurality of data items to the backup storage without copying multiple copies of an individual data item to the backup storage.
17 . A method comprising:
maintaining a cluster on a plurality of nodes of a virtualization system, the cluster having a distributed metadata system that uses a persistent data structure to store system metadata, wherein a storage pool of the cluster is managed using the system metadata, the system metadata is stored on multiple nodes of the plurality of nodes, the system metadata comprises a plurality of portions of metadata, individual portions of the plurality of portions of metadata are stored as both a primary copy on a node of the multiple nodes and at least one redundant copy on at least one other node of the multiple nodes, and at least two nodes of the multiple nodes store both a primary copy of at least one individual portion and a redundant copy of at least one other individual portion;
determining that the system metadata is to be migrated to a backup storage that is external to the cluster; and
migrating the system metadata to the backup storage at least by:
identifying and selecting, from the primary and redundant copies, a single copy of the individual portions of the plurality of portions of metadata for transfer to the backup storage, wherein the primary copy of the individual portions of the plurality of portions of metadata is selected as the single copy of the individual portions of the plurality of portions of metadata, and
transmitting the identified and selected single copy of the individual portions of the plurality of portions of metadata from the multiple nodes to the backup storage.
18 . The method of claim 17 , wherein the multiple nodes transfer different portions of the system metadata to the backup storage, and the different portions comprise different entries in the persistent data structure.
19 . The method of claim 17 , wherein the primary copies of portions of the system metadata comprise different entries in the persistent data structure that are transferred by the multiple nodes using multiple connections on each of the multiple nodes.
20 . The method of claim 17 , wherein the persistent data structure comprises a sorted string table (SST) and the SST is divided into multiple ranges of SST entries using a column family and token range.
21 . The method of claim 20 , wherein the multiple ranges of SST entries are further divided into multiple subsets of SST entries by the multiple nodes using the column family and token ranges.
22 . The method of claim 21 , further comprises, prior to migration, compacting the multiple subsets of SST entries, generating snapshots of respective ones of the multiple subsets of SST entries, or validating the multiple subsets of SST entries.
23 . The method of claim 17 , wherein the backup storage comprises a cloud disk and the storage pool is constructed from a plurality of storage devices directly attached to respective nodes of the plurality of nodes.
24 . The method of claim 17 , further comprises:
identifying a plurality of data items having a corresponding redundant copy on the storage pool;
selecting a single copy of each of the plurality of data items for transfer to the backup storage; and migrating the single copy of each of the plurality of data items to the backup storage without copying multiple copies of an individual data item to the backup storage.