Table placement in distributed databases
A node type of a plurality of distributed nodes to which a table to be added to a distributed database should be assigned can be identified by applying a set of placement rules defined for the table. The set of placement rules can also be applied to determine whether the table should be partitioned into more than one partition. A table group name associated with the table can be obtained and used in conjunction with the node type and determination of whether to partition the table to store the table in the distributed database on at least one node of the plurality of nodes as one or more partitions.
1. A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
identifying a table stored across a plurality of partitions in a distributed database, the distributed database comprising a plurality of tables located on a plurality of distributed nodes;
detecting that an average record count of the plurality of partitions is at or above a record threshold;
determining that a number of available nodes in the plurality of distributed nodes is at or above a node threshold, wherein the node threshold is equal to at least twice a number of partitions in the plurality of partitions;
dividing, based on classification information for the table and a placement rule defined for the table, at least one of the plurality of partitions into more than one partition by creating a new partition for the at least one of the plurality of partitions; and
placing, based on the classification information and the placement rule, the new partition onto at least a portion of the available nodes.
2. A computer program product as in claim 1 , wherein the record threshold is specified in a set of placement rules including the placement rule defined for the table, wherein the classification information includes a schema name, a table name, and/or an application type, and wherein the set of placement rules includes partitioning information.
3. A computer program product as in claim 1 , wherein the dividing comprises dividing each of the plurality of partitions into two new partitions, and wherein the operations further comprise placing each of the new partitions onto at least a portion of the available nodes.
4. A computer program product as in claim 3 , wherein the operations further comprise:
identifying a server type for the table, wherein the available nodes are only determined to be available when they are of the server type, and wherein each the new partitions are placed only onto nodes having the server type.
5. A system comprising:
computer circuitry configured to perform operations comprising:
identifying a table stored across a plurality of partitions in a distributed database, the distributed database comprising a plurality of tables located on a plurality of distributed nodes;
detecting that an average record count of the plurality of partitions is at or above a record threshold;
determining that a number of available nodes in the plurality of distributed nodes is at or above a node threshold, wherein the node threshold is equal to at least twice a number of partitions in the plurality of partitions;
dividing, based on classification information for the table and a placement rule defined for the table, at least one of the plurality of partitions into more than one partition by creating a new partition for the at least one of the plurality of partitions; and
placing, based on the classification information and the placement rule, the new partition onto at least a portion of the available nodes.
6. A system as in claim 5 , wherein the record threshold is specified in a set of placement rules including the placement rule defined for the table, wherein the classification information includes a schema name, a table name, and/or an application type, and wherein the set of placement rules includes partitioning information.
7. A system as in claim 5 , wherein the dividing comprises dividing each of the plurality of partitions into two new partitions, and wherein the operations further comprise placing each of the new partitions onto at least a portion of the available nodes.
8. A system as in claim 7 , wherein the operations further comprise:
identifying a server type for the table, wherein the available nodes are only determined to be available when they are of the server type, and wherein each the new partitions are placed only onto nodes having the server type.
9. A system as in claim 5 , wherein the computer circuitry comprises:
at least one programmable processor; and
a machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform the operations.
10. A computer-implemented method comprising:
identifying a table stored across a plurality of partitions in a distributed database, the distributed database comprising a plurality of tables located on a plurality of distributed nodes;
detecting that an average record count of the plurality of partitions is at or above a record threshold;
determining that a number of available nodes in the plurality of distributed nodes is at or above a node threshold, wherein the node threshold is equal to at least twice a number of partitions in the plurality of partitions;
dividing, based on classification information for the table and a placement rule defined for the table, at least one of the plurality of partitions into more than one partition by creating a new partition for the at least one of the plurality of partitions; and
placing, based on the classification information and the placement rule, the new partition onto at least a portion of the available nodes.
11. A computer-implemented method as in claim 10 , wherein the record threshold is specified in a set of placement rules including the placement rule defined for the table, wherein the classification information includes a schema name, a table name, and/or an application type, and wherein the set of placement rules includes partitioning information.
12. A computer-implemented method as in claim 10 , wherein the dividing comprises dividing each of the plurality of partitions into two new partitions, and wherein the method further comprises placing each of the new partitions onto at least a portion of the available nodes.
13. A computer-implemented method as in claim 12 , further comprising:
identifying a server type for the table, wherein the available nodes are only determined to be available when they are of the server type, and wherein each the new partitions are placed only onto nodes having the server type.
14. A computer-implemented method as in claim 10 , wherein the identifying, the identifying, detecting, determining, dividing, and placing are performed by at least one system comprising computer circuitry.