Workload management in distributed database systems
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for instrumentation and control of workloads in a massively parallel processing database. Deployment is in a cluster which mirrors the cluster of the database to be controlled. The system includes data publishing modules, action agents, rules processing modules, deployment managers, rule compilation and management tools. Together these provide a distributed, fault tolerant system for the automated rules-based control of work in a database cluster. A rule specifies an action on the system and a condition for performing the action, the condition including a value, one or more tags corresponding to the value, and a relation between the one or more tags and the value. The system triggers the action based on evaluating the condition.
1. A method, comprising:
executing, by a distributed database system, a received query resulting in retrieval of one or more data items into one or more nodes of the distributed database system, each node corresponding to a physical server;
receiving a rule by a rule processing module on any one or the one or more nodes, the rule specifying a query resource modification action on the database system and a condition for performing the query resource modification action, the condition including a first value, one or more tags corresponding to the first value, and a relation between the one or more tags and the first value;
publishing, by a data publishing module of the database system, the one or more data items, each data item being a self-describing data item that includes data and metadata describing the data, the metadata comprising a respective second value and one or more data scopes associated with the respective second value, each data scope representing a respective context of the data;
receiving, by the rule processing module, as a subscriber to the data publishing module, some or all of the published one or more data items;
evaluating the condition by the rule processing module to determine whether the condition is satisfied for each of the received one or more data items based on the respective metadata, wherein the evaluation comprises, for each data item;
determining that the one or more tags corresponding to one or more data scopes of the condition match the one or more data scopes of the data item; and
determining that the first value of the condition and the second value of the data item satisfy the relation of the condition; and
triggering the database system to perform the query resource modification action, thereby causing the database system to modify resource usage devoted to the query in response to determining that the condition is satisfied.
2. The method of claim 1 , wherein receiving the rule is triggered by an add rule command, the add rule command specifying whether the rule is transient or persistent, wherein a persistent rule remains active until deleted, and a transient rule stops being active upon a shutdown of the rule processing module.
3. The method of claim 1 , wherein the action has an action scope specifying whether the action is a global action to be performed on each node of the database system or a local action to be performed on a specific node of the database system.
4. The method of claim 1 , wherein the action comprises at least one of displaying resource usage status of the database system, displaying a representation of data in the data items, globally cancelling a query, locally cancelling a query, or throttling a query.
5. The method of claim 1 , further comprising:
the rule processing module subscribing from the data publishing module to data items that have an identifier that matches at least one of the one or more tags.
6. The method of claim 1 , wherein the relation includes at least one of a comparison or a regular expression match.
7. A non-transitory storage device storing instructions that, when executed by one or more computer processors, are operable cause one or more processors to perform operations comprising:
executing, by a distributed database system, a received query resulting in retrieval of one or more data items into one or more nodes of the distributed database system, each node corresponding to a physical server;
receiving a rule by a rule processing module on any one or the one or more nodes, the rule specifying a query resource modification action on the database system and a condition for performing the query resource modification action, the condition including a first value, one or more tags corresponding to the first value, and a relation between the one or more tags and the first value;
publishing, by a data publishing module of the database system, the one or more data items, each data item being a self-describing data item that includes data and metadata describing the data, the metadata comprising a respective second value and one or more data scopes associated with the respective second value, each data scope representing a respective context of the data;
receiving, by the rule processing module, as a subscriber to the data publishing module, some or all of the published one or more data items;
evaluating the condition by the rule processing module to determine whether the condition is satisfied for each of the received one or more data items based on the respective metadata, wherein the evaluation comprises, for each data item:
determining that the one or more tags corresponding to one or more data scopes of the condition match the one or more data scopes of the data item; and
determining that the first value of the condition and the second value of the data item satisfy the relation of the condition; and
triggering the database system to perform the query resource modification action, thereby causing the database system to modify resource usage devoted to the query in response to determining that the condition is satisfied.
8. The non-transitory storage device of claim 7 , wherein receiving the rule is triggered by an add rule command, the add rule command specifying whether the rule is transient or persistent, wherein a persistent rule remains active until deleted, and a transient rule stops being active upon a shutdown of the rule processing module.
9. The non-transitory storage device of claim 7 , wherein the action has an action scope specifying whether the action is a global action to be performed on each node of the database system or a local action to be performed on a specific node of the database system.
10. The non-transitory storage device of claim 7 , wherein the action comprises at least one of displaying resource usage status of the database system, displaying a representation of data in the data items, globally cancelling a query, locally cancelling a query, or throttling a query.
11. The non-transitory storage device of claim 7 , the operations further comprising:
the rule processing module subscribing from the data publishing module to data items that have an identifier that matches at least one of the one or more tags.
12. The non-transitory storage device of claim 7 , wherein the relation includes at least one of a comparison or a regular expression match.
13. A system comprising:
one or more processors; and
a non-transitory storage device storing instructions that, when executed by one or more computer processors, are operable cause one or more processors to perform operations comprising:
executing, by a distributed database system, a received query resulting in retrieval of one or more data items into one or more nodes of the distributed database system, each node corresponding to a physical server;
receiving a rule by a rule processing module on any one or the one or more nodes, the rule specifying a query resource modification action on the database system and a condition for performing the query resource modification action, the condition including a first value, one or more tags corresponding to the first value, and a relation between the one or more tags and the first value;
publishing, by a data publishing module of the database system, the one or more data items, each data item being a self-describing data item that includes data and metadata describing the data, the metadata comprising a respective second value and one or more data scopes associated with the respective second value, each data scope representing a respective context of the data;
receiving, by the rule processing module, as a subscriber to the data publishing module, some or all of the published one or more data items;
evaluating the condition by the rule processing module to determine whether the condition is satisfied for each of the received one or more data items based on the respective metadata, wherein the evaluation comprises, for each data item:
determining that the one or more tags corresponding to one or more data scopes of the condition match the one or more data scopes of the data item; and
determining that the first value of the condition and the second value of the data item satisfy the relation of the condition; and
triggering the database system to perform the query resource modification action, thereby causing the database system to modify resource usage devoted to the query in response to determining that the condition is satisfied.
14. The system of claim 13 , wherein receiving the rule is triggered by an add rule command, the add rule command specifying whether the rule is transient or persistent, wherein a persistent rule remains active until deleted, and a transient rule stops being active upon a shutdown of the rule processing module.
15. The system of claim 13 , wherein the action has an action scope specifying whether the action is a global action to be performed on each node of the database system or a local action to be performed on a specific node of the database system.
16. The system of claim 13 , wherein the action comprises at least one of displaying resource usage status of the database system, displaying a representation of data in the data items, globally cancelling a query, locally cancelling a query, or throttling a query.
17. The system of claim 13 , wherein the relation includes at least one of a comparison or a regular expression match.