Cluster bootstrapping for distributed computing systems
A distributed computing system is described that leverages a nearline storage layer to minimize the downtime required for bootstrapping a new computing cluster in the distributed computing system. The system executes a computing cluster comprising a set of computing nodes and determines a set of one or more data segments to be written to a nearline storage system. The system writes the data segments to the nearline storage system. In certain examples, the system receives a request to create a second computing cluster and responsive to the request, bootstraps the second computing cluster using the set of data segments stored on the nearline storage system. The system additionally leverages the nearline storage layer to accelerate query processing by the computing nodes of a computing cluster.
1. A method, comprising:
executing, by a distributed computing system providing a data processing service, a first computing cluster comprising a first set of one or more computing nodes;
identifying, by a computing node in the first set of one or more computing nodes of the distributed computing system, a plurality of data segments stored in a cache associated with the computing node, the cache comprising a set of one or more cache memory tiers;
selecting, by the computing node in the first set of one or more computing nodes of the distributed computing system, a set of one or more data segments from the plurality of data segments to be moved to a nearline storage cache tier, the set of one or more cache memory tiers comprising the nearline storage cache tier;
determining, by the distributed computing system, a subset of one or more data segments from the set of one or more data segments to be moved to a nearline storage system associated with the distributed computing system;
based at least in part upon the determining, writing, by the computing node in the first set of one or more computing nodes of the distributed computing system, the subset of one or more data segments to the nearline storage system;
receiving, by the distributed computing system, a request to create a second computing cluster in the distributed computing system, the second computing cluster comprising a second set of one or more computing nodes; and
responsive to the request, bootstrapping, by the distributed computing system, the second computing cluster using the subset of one or more data segments stored in the nearline storage system.
2. The method of claim 1 , wherein the request to create the second computing cluster is received responsive to at least one of a cluster upgrade, a cluster migration, or a cluster failover associated with the first computing cluster.
3. The method of claim 1 , wherein bootstrapping, by the distributed computing system, the second computing cluster comprises:
obtaining, by one or more computing nodes in the second set of computing nodes comprising the second computing cluster, the subset of one or more data segments stored in the nearline storage system; and
pre-populating, by the one or more computing nodes, a cache associated with the one or more computing nodes in the second set of computing nodes with the subset of one or more data segments.
4. The method of claim 3 , wherein the nearline storage system represents an intermediary storage layer between the cache associated with the one or more computing nodes in the second set of computing nodes and an object storage system associated with the distributed computing system.
5. The method of claim 1 , further comprising wherein:
receiving, by the computing node in the first set of computing nodes comprising the first computing cluster, a query for execution;
determining, by the computing node, that one or more data segments for executing the query are present in the cache associated with the computing node;
executing, by the computing node, the query using the one or more data segments;
obtaining, by the computing node, one or more updated data segments as a result of executing the query; and
writing, by the computing node, the one or more updated data segments to the nearline storage system associated with the distributed computing system.
6. The method of claim 1 , further comprising
receiving, by the computing node in the first set of computing nodes comprising the first computing cluster, a query for execution;
determining, by the computing node, that one or more data segments for executing the query are not present in the cache associated with the computing node;
responsive to determining that the one or more data segments are not present in the cache, obtaining, by the computing node, the one or more data segments from an object storage system associated with the distributed computing system; and
writing, by the computing node, the one or more data segments to the nearline storage system associated with the distributed computing system.
7. The method of claim 1 , wherein selecting, by the computing node, the set of one or more data segments from the plurality of data segments is performed using a selection technique, wherein the selection technique comprises at least one of a least recently used (LRU) selection technique or a least frequently used (LFU) selection technique.
8. The method of claim 1 , wherein the set of one or more cache memory tiers comprises an object storage cache tier.
9. The method of claim 8 , wherein the object storage cache tier is mapped to an object storage system associated with the distributed computing system, wherein the object storage cache tier is configured to temporarily store the plurality of data segments for the object storage system.
10. The method of claim 1 , wherein the nearline storage cache tier is mapped to a nearline storage system associated with the distributed computing system, wherein the nearline storage cache tier is configured to temporarily store the set of one or more data segments for the nearline storage system.
11. The method of claim 1 , wherein the first computing cluster and the second computing cluster are associated with a first tenant of the distributed computing system.
12. A distributed computing system providing a data processing service comprising:
a memory; and
one or more processors configured to perform processing, the processing comprising:
executing, by the distributed computing system, a first computing cluster comprising a first set of one or more computing nodes;
identifying, by a computing node in the first set of one or more computing nodes of the distributed computing system, a plurality of data segments stored in a cache associated with the computing node, the cache comprising a set of one or more cache memory tiers;
selecting, by the computing node in the first set of one or more computing nodes of the distributed computing system, a set of one or more data segments from the plurality of data segments to be moved to a nearline storage cache tier, the set of one or more cache memory tiers comprising the nearline storage cache tier;
determining, by the distributed computing system, a subset of one or more data segments from the set of one or more data segments to be moved to a nearline storage system associated with the distributed computing system;
based at least in part upon the determining, writing, by the computing node in the first set of one or more computing nodes of the distributed computing system, the subset of one or more data segments to the nearline storage system;
receiving, by the distributed computing system, a request to create a second computing cluster in the distributed computing system, the second computing cluster comprising a second set of one or more computing nodes; and
responsive to the request, bootstrapping, by the distributed computing system, the second computing cluster using the subset of one or more data segments stored in the nearline storage system.
13. The system of claim 12 , wherein the request to create the second computing cluster is received responsive to at least one of a cluster upgrade, a cluster migration, or a cluster failover associated with the first computing cluster.
14. The system of claim 12 , wherein bootstrapping, by the distributed computing system, the second computing cluster comprises:
obtaining, by one or more computing nodes in the second set of computing nodes comprising the second computing cluster, the subset of one or more data segments stored in the nearline storage system; and
pre-populating, by the one or more computing nodes, a cache associated with the one or more computing nodes in the second set of computing nodes with the subset of one or more data segments.
15. A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices for performing operations comprising:
executing, by a distributed computing system, a first computing cluster comprising a first set of one or more computing nodes;
identifying, by a computing node in the first set of one or more computing nodes of the distributed computing system, a plurality of data segments stored in a cache associated with the computing node, the cache comprising a set of one or more cache memory tiers;
selecting, by the computing node in the first set of one or more computing nodes of the distributed computing system, a set of one or more data segments from the plurality of data segments to be moved to a nearline storage cache tier, the set of one or more cache memory tiers comprising the nearline storage cache tier;
determining, by the distributed computing system, a subset of one or more data segments from the set of one or more data segments to be moved to a nearline storage system associated with the distributed computing system;
based at least in part upon the determining, writing, by the computing node in the first set of one or more computing nodes of the distributed computing system, the subset of one or more data segments to the nearline storage system;
receiving a request to create a second computing cluster in the distributed computing system, the second computing cluster comprising a second set of one or more computing nodes; and
responsive to the request, bootstrapping the second computing cluster using the subset of one or more data segments stored in the nearline storage system.
16. The non-transitory computer-readable medium of claim 15 , further comprising:
receiving, by the computing node in a first set of computing nodes comprising a first computing cluster, a query for execution;
determining, by the computing node, that one or more data segments for executing the query are present in the cache associated with the computing node;
executing, by the computing node, the query using the one or more data segments;
obtaining, by the computing node, one or more updated data segments as a result of executing the query; and
writing, by the computing node, the one or more updated data segments to the nearline storage system associated with the distributed computing system.