Monitoring and alerting mechanisms
Computing systems methods, and non-transitory storage media are provided for receiving a monitoring request. The monitoring request includes one or more entities or attributes to be monitored, one or more rules to be evaluated with respect to the entities or attributes, and one or more downstream actions to be selectively triggered based on the evaluation. Next, data regarding the entities or the attributes is obtained. Next, a log is generated. The log includes changes or updates, relative to a previous iteration, of the entities or the attributes. The changes or updates correspond to the rules. Next, the changes or the updates are evaluated against the one or more rules and based on the log. Next, one or more actions are selectively implemented based on the evaluation of the changes or the updates.
1 . A computing system, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to perform:
receiving a monitoring request, wherein the monitoring request comprises entities or attributes, to be monitored, rules to evaluate whether the entities or the attributes are conforming or nonconforming, or a level of conformity, and one or more actions to be selectively triggered based on the evaluation, the entities or attributes comprising a first entity or a first attribute of the first entity, and a second entity or a second attribute of the second entity, the entities or attributes being obtained, derived, or inferred from an input dataset, the rules comprising a first rule for evaluating the first entity or the first attribute and a second rule for evaluating the second entity or the second attribute, the first rule being semantically native to a first data type and the second rule being semantically native to a second data type, the second rule being dynamically changed depending on a first level of conformity of the first entity or the first attribute according to the first rule;
obtaining event data associated with the entities or the attributes, wherein the event data comprises sequences of events associated with the entities or the attributes;
generating a first converted dataset comprising at least a first subset of the event data associated with the first entity or the first attribute, and generating a second converted dataset comprising at least a second subset of the event data associated with the second entity or the second attribute, wherein the first converted dataset has a first format consistent with the first data type and the second converted dataset has a second format consistent with the second data type to expedite evaluation of the rules;
evaluating a first level of conformity or nonconformity of the first converted dataset according to the first rule; and
evaluating a second level of conformity or nonconformity of the second converted dataset according to the second rule;
based on the first and second levels of conformity or nonconformity, determining any level of an alert to be triggered;
selectively triggering the alert based on the determined level of the alert; and
performing a downstream action associated with the determined level of the alert.
2 . The computing system of claim 1 , wherein the evaluating of the level of conformity or nonconformity is based on one or more access constraints of the entities or attributes from different sections of the database.
3 . The computing system of claim 1 , wherein the downstream action comprises adjusting a sampling rate of the monitoring based on the determined level of the alert.
4 . The computing system of claim 1 , wherein the evaluating of the level of conformity or nonconformity is based on a join key, a primary key, or a foreign key constraint associated with the event data.
5 . The computing system of claim 1 , wherein:
the evaluating of the level of conformity comprises evaluating, using different computing nodes, the respective attributes.
6 . The computing system of claim 1 , wherein the one or more actions comprise an alert or a notification to an entity that transmitted the monitoring request.
7 . The computing system of claim 1 , wherein the one or more actions comprise:
writing captured data to a database or an external database.
8 . The computing system of claim 1 , further comprising the database storing the one or more entities or attributes are obtained, the database being directly integrated with the computing system, and the evaluating of the level of conformity or nonconformity of the event data to the one or more rules, the determining of the any level of the alert to be triggered, and the selectively triggering of the alert, are performed within the database.
9 . The computing system of claim 8 , wherein the database is directly integrated with the computing system at a back end of the database, wherein the back end corresponds to a data access layer.
10 . The computing system of claim 8 , wherein the database resides on a database layer, the database layer being a separate layer from a metadata layer that stores relationship data of objects within the database, the relationship data comprising a join key, a primary key, a foreign key constraint, or a different key.
11 . The computing system of claim 1 , wherein performing a downstream action associated with the determined level of the alert is in response to simulating the downstream action.
12 . The computing system of claim 1 , wherein evaluating a first level of conformity or nonconformity or evaluating a second level of conformity or nonconformity of the second converted dataset is based on one or more changes of a join key, a primary key, or a foreign key constraint.
13 . The computing system of claim 1 , wherein evaluating a first level of conformity or nonconformity is performed by a first computing node and evaluating a second level of conformity or nonconformity is based on a second computing node.
14 . The computing system of claim 1 , wherein performing a downstream action comprises:
determining whether the first level of conformity or nonconformity, or determining whether the second level of conformity or nonconformity causes a trigger condition to be satisfied or unsatisfied;
predicting whether one or more additional changes in the first level of conformity or nonconformity or in the second level of conformity or nonconformity causes the trigger condition to be satisfied or unsatisfied;
in response to the first level of conformity or nonconformity or the second level of conformity or nonconformity causing the trigger condition to be satisfied and the additional changes causing the trigger condition to be unsatisfied, implementing a first action; and
in response to the first level of conformity or nonconformity or the second level of conformity or nonconformity causing the trigger condition to be satisfied and the trigger condition remaining satisfied after the additional changes, implementing a second action different from the first action.
15 . The computing system of claim 1 , wherein performing a downstream action comprises:
determining whether the first level of conformity or nonconformity, or determining whether the second level of conformity or nonconformity causes a trigger condition to be satisfied or unsatisfied;
predicting whether one or more additional changes in the first level of conformity or nonconformity or in the second level of conformity or nonconformity causes the trigger condition to be satisfied or unsatisfied;
in response to the first level of conformity or nonconformity or the second level of conformity or nonconformity causing the trigger condition to be unsatisfied and the additional changes causing the trigger condition to be satisfied, implementing a first action; and
in response to the first level of conformity or nonconformity or the second level of conformity or nonconformity causing the trigger condition to be satisfied and the trigger condition remaining satisfied after the additional changes, implementing a second action different from the first action.
16 . The computing system of claim 1 , wherein the computing system comprises a distributed computing system having a first processor, a second processor and a third processor, wherein the first processor is configured to evaluate the first entity or the first attribute, the second processor is configured to evaluate the second entity or the second attribute, and the third processor is configured to evaluate one or more additional rules with respect to a combination of the first entity or the first attribute, and the second entity or the second attribute.
17 . The computing system of claim 1 , wherein the evaluating comprises determining an anomaly status in response to determining that a first rate of change or a first variance in the first attribute deviates by more than a threshold amount from a second rate of change or a second variable in the second attribute; and the selectively implementing the one or more actions comprises implementing a first action in response to determining the anomaly status and refraining from implementing the first action in response to determining a non-anomaly status.
18 . A computer-implemented method of a computing system, the computer-implemented method comprising:
receiving a monitoring request, wherein the monitoring request comprises entities or attributes, to be monitored, rules to evaluate whether the entities or the attributes are conforming or nonconforming, or a level of conformity, and one or more actions to be selectively triggered based on the evaluation, the entities or attributes comprising a first entity or a first attribute of the first entity, and a second entity or a second attribute of the second entity, the rules comprising a first rule for evaluating the first entity or the first attribute and a second rule for evaluating the second entity or the second attribute, the first rule being semantically native to a first data type and the second rule being semantically native to a second data type, the second rule being dynamically changed depending on a first level of conformity of the first entity or the first attribute according to the first rule;
obtaining event data associated with the entities or the attributes, wherein the event data comprises sequences of events associated with the entities or the attributes;
generating a first converted dataset comprising at least a first subset of the event data associated with the first entity or the first attribute, and generating a second converted dataset comprising at least a second subset of the event data associated with the second entity or the second attribute, wherein the first converted dataset has a first format consistent with the first data type and the second converted dataset has a second format consistent with the second data type to expedite evaluation of the rules;
evaluating a first level of conformity or nonconformity of the first converted dataset according to the first rule; and
evaluating a second level of conformity or nonconformity of the second converted dataset according to the second rule;
based on the first and second levels of conformity or nonconformity, determining any level of an alert to be triggered;
selectively triggering the alert based on the determined level of the alert; and
performing a downstream action associated with the determined level of the alert.
19 . The computer-implemented method of claim 18 , wherein the evaluating of the level of conformity or nonconformity is based on one or more access constraints of the entities or attributes from different sections of the database.
20 . The computer-implemented method of claim 18 , wherein the downstream action comprises adjusting a sampling rate of the monitoring based on the determined level of the alert.