Selecting subscribing computing node to execute data storage plan for data shard

1. A non-transitory computer-readable data storage medium storing program code executable by a primary computing node of a distributed database system to perform processing, the distributed database system maintaining a database that includes a data shard for which the primary computing node is responsible, the processing comprising:

receiving or generating a data storage plan for the data shard, the data storage plan identifying a file subset of a plurality of data storage files of the data shard to be merged into a larger data storage file, the data storage plan further identifying a node subset of a plurality of computing nodes of the distributed database system subscribing to the data shard, the subset including the primary computing node;

identifying which computing nodes of the node subset each have a current utilization satisfying an availability criterion, as candidate computing nodes, comprising:

identifying a configuration of each computing node of the node subset within a resource pool, the configuration specifying the current utilization of the computing node; and

selecting each computing node of which the current utilization satisfies the availability criterion, as one of the candidate computing nodes;

identifying which data storage files of the file subset each candidate computing node locally caches, comprising:

sending a message to the candidate computing node to receive information regarding the data storage files of the file subset that the candidate computing node locally caches; and

receiving a reply from the candidate computing node including the information regarding the data storage files of the file subset that the candidate computing node locally caches;

selecting one of the candidate computing nodes to execute the data storage plan, based on the data storage files of the file subset that each candidate computing node locally caches; and

causing the selected one of the candidate computing nodes to execute the data storage plan by merging the data storage files of the file subset into the larger data storage file and uploading the larger data storage file to a global shared storage accessible by the computing nodes of the distributed database system,

wherein execution of the data storage plan resolves fragmentation of data stored in the data shard resulting from queries being performed against the data.

2. The non-transitory computer-readable data storage medium of claim 1 , wherein the information comprises one or more of:

a number of the data storage files of the file subset that the candidate computing node locally caches; and

a size of the data storage files of the file subset that the candidate computing node locally caches.

3. The non-transitory computer-readable data storage medium of claim 1 , wherein selecting the one of the candidate computing nodes to execute the data storage plan, based on the data storage files of the file subset that each candidate computing node locally caches, comprises:

selecting the one of the candidate computing nodes to execute the data storage plan as the candidate computing node locally caching a greatest number of the data storage files of the file subset.

4. The non-transitory computer-readable data storage medium of claim 1 , wherein selecting the one of the candidate computing nodes to execute the data storage plan, based on the data storage files of the file subset that each candidate computing node locally caches, comprises:

selecting the one of the candidate computing nodes to execute the data storage plan as the candidate computing node locally caching a largest size of the data storage files of the file subset.

5. The non-transitory computer-readable data storage medium of claim 1 , wherein selecting the one of the candidate computing nodes to execute the data storage plan, based on the data storage files of the file subset that each candidate computing node locally caches, comprises:

for each candidate computing node, computing a score based on a number of the data storage files that the candidate computing node locally caches and based on a size of the data storage files that the candidate computing node locally caches;

selecting the one of the candidate computing nodes to execute the data storage plan as the candidate computing node having a highest score.

6. The non-transitory computer-readable data storage medium of claim 1 , wherein the selected one of the candidate computing nodes to execute the data storage plan is other than the primary computing node, and wherein causing the selected one of the candidate computing nodes to execute the data storage plan comprises:

sending the data storage plan to the selected one of the candidate computing nodes for execution.

7. The non-transitory computer-readable data storage medium of claim 6 , wherein the processing further comprises:

receiving data storage plan execution results from the selected one of the candidate computing nodes after execution of the data storage plan; and

performing further operations regarding the data shard, based on the data storage plan execution results.

8. The non-transitory computer-readable data storage medium of claim 1 , wherein the selected one of the candidate computing nodes to execute the data storage plan is the primary computing node, and wherein causing the selected one of the candidate computing nodes to execute the data storage plan comprises:

executing the data storage plan.

9. A method comprising:

receiving, by a computing node of a distributed database system maintaining a database including a data shard to which a node subset of a plurality of computing nodes of the distributed database system including the computing node and a primary computing node responsible for the data shard, a data storage plan identifying the node subset and a file subset of a plurality of data storage files of the data shard to be merged into a larger data storage file, the data storage plan received from the primary computing node;

executing, by the computing node, the data storage plan by merging the data storage files of the file subset into the larger data storage file and uploading the larger data storage file to a global shared storage accessible by the computing nodes of the distributed database system,

wherein execution of the data storage plan resolves fragmentation of data stored in the data shard resulting from queries being performed against the data,

and wherein the primary computing node:

identifies which computing nodes of the node subset each have a current utilization satisfying an availability criterion, as candidate computing nodes, the candidate computing nodes including the computing node, by:

identifying a configuration of each computing node of the node subset within a resource pool, the configuration specifying the current utilization of the computing node; and

selecting each computing node of which the current utilization satisfies the availability criterion, as one of the candidate computing nodes;

identifies which data storage files of the file subset each candidate computing node locally caches, comprising:

sending a message to the candidate computing node to receive information regarding the data storage files of the file subset that the candidate computing node locally caches; and

receiving a reply from the candidate computing node including the information regarding the data storage files of the file subset that the candidate computing node locally caches; and

selects the computing node to execute the data storage plan, based on the data storage files of the file subset that each candidate computing node locally cache.

10. The method of claim 9 , further comprising:

receiving, by the computing node from the primary computing node, a message to send information regarding the data storage files of the file subset that the computing node locally caches; and

sending, by the computing node to the primary computing node, a reply including the information regarding the data storage files of the file subset that the computing node locally caches.

11. The method of claim 10 , wherein the information comprises:

a number of the data storage files of the file subset that the computing node locally caches.

12. The method of claim 10 , wherein the information comprises a size of the data storage files of the file subset that the candidate computing node locally caches.

13. A distributed database system comprising:

a plurality of computing nodes; and

a global storage storing a plurality of data shards of a database, each data shard including a plurality of data storage files and for which one of the computing nodes is a primary computing node responsible for the data shard,

wherein the primary computing node for each data shard is to:

receive or generate a data storage plan for the data shard, the data storage plan identifying a file subset of the data storage files of the data shard to be merged into a larger data storage file, the data storage plan further identifying a node subset of the computing nodes subscribing to the data shard, the subset including the primary computing node;

identify which computing nodes of the node subset each have a current utilization satisfying an availability criterion, as candidate computing nodes, by:

identifying a configuration of each computing node of the node subset within a resource pool, the configuration specifying the current utilization of the computing node; and

selecting each computing node of which the current utilization satisfies the availability criterion, as one of the candidate computing nodes;

identify which data storage files of the file subset each candidate computing node locally caches, by:

sending a message to the candidate computing node to receive information regarding the data storage files of the file subset that the candidate computing node locally caches; and

receiving a reply from the candidate computing node including the information regarding the data storage files of the file subset that the candidate computing node locally caches;

select one of the candidate computing nodes to execute the data storage plan, based on the data storage files of the file subset that each candidate computing node locally caches; and

cause the selected one of the candidate computing nodes to execute the data storage plan by merging the data storage files of the file subset into the larger data storage file and uploading the larger data storage file to a global shared storage accessible by the computing nodes of the distributed database system,

wherein execution of the data storage plan resolves fragmentation of data stored in the data shard resulting from queries being performed against the data.

14. The distributed database system of claim 13 , wherein the information comprises a number of the data storage files of the file subset that the candidate computing node locally caches.

15. The distributed database system of claim 13 , wherein the information comprises a size of the data storage files of the file subset that the candidate computing node locally caches.

16. The distributed database system of claim 13 , wherein the primary node for each data shard is to select the one of the candidate computing nodes to execute the data storage plan, based on the data storage files of the file subset that each candidate computing node locally caches, by:

for each candidate computing node, computing a score based on a number of the data storage files that the candidate computing node locally caches and/or based on a size of the data storage files that the candidate computing node locally caches; and

selecting the one of the candidate computing nodes to execute the data storage plan as the candidate computing node having a highest score.