Event processing in a distributed computing environment
A method distributes event processing in a distributed computing environment by storing transmission costs between computational node pairs, determining a low-cost route to deliver events from producers to consumers based on the costs, applying an augmentation to events via the low-cost route to generate augmented events, and delivering the augmented events to consumers via the low-cost route. Specifically, the method involves storing transmission costs between each of multiple computational node pairs in the distributed environment, where nodes comprise event producers and consumers. A low-cost route is determined to deliver a detected event from a producer to a consumer based on the stored costs. An augmentation is applied to the event via the low-cost route to generate an augmented event. The augmented event is then delivered to the consumer via the same low-cost route.
1. A method to distribute event processing in a distributed computing environment, the method comprising:
storing transmission costs between each of a plurality of pairs of computational nodes of a plurality of computational nodes in the distributed computing environment, the plurality of computational nodes comprising event producers and event consumers;
determining a low-cost route to deliver an event detected at an event producer to an event consumer based on the transmission costs;
applying, via the low-cost route, an augmentation to the event to generate an augmented event; and
delivering, via the low-cost route, the augmented event to the event consumer.
2. The method of claim 1 , wherein determining the low-cost route comprises:
determining a direct route between the event producer and the event consumer;
comparing a cost of the direct route to a threshold; and
in response to the cost of the direct route transgressing the threshold, determining a route comprising a node intermediate to the event producer and the event consumer, a cost of the route being lower than the cost of the direct route, the cost of the route being based on transmission cost between the event producer and intermediate node and transmission cost between the intermediate node and the event consumer.
3. The method of claim 1 , further comprising generating an event consumer cohort, the event consumer cohort comprising a first event consumer and a second event consumer, a transmission cost between the first event consumer and the second event consumer being below a threshold.
4. The method of claim 3 , further comprising:
provisioning an event consolidator on a node of the plurality of computational nodes with a lowest total transmission cost to the event consumer cohort, total transmission costs to be computed based on transmission costs between the node and each of the event consumers in the event consumer cohort; and
configuring the event consolidator to receive the event from the event producer and deliver the event to members of the event consumer cohort.
5. The method of claim 1 , further comprising instantiating a plurality of mesh agents that are provisioned to support applying the augmentation to the event using the plurality of computational nodes within the distributed computing environment.
6. The method of claim 5 , further comprising deploying the plurality of mesh agents as a full mesh network or a partial mesh network among the plurality of computational nodes.
7. The method of claim 1 , wherein the distributed computing environment comprises a plurality of networks and the method further comprises identifying the low-cost route across the plurality of networks.
8. The method of claim 1 , wherein the augmentation comprises at least one transformation, filtering, correlation, contextualization or analytics operation.
9. The method of claim 1 , wherein:
the event is associated with a unique identifier, at least one event producer of the event, and at least one event consumer of the event; and
the event producers and the event consumers are associated with locations corresponding to computational nodes of the plurality of computational nodes.
10. The method of claim 9 , wherein the method further stores event consolidator information with respect to the event producers and the event consumers.
11. The method of claim 8 , wherein the augmentation has a location dependency, the location dependency restricting the augmentation to a respective specified node of the plurality of computational nodes.
12. The method of claim 6 , wherein the deploying of the plurality of mesh agents comprises using augmentation provisioning rules to determine locations corresponding to computational nodes for selected mesh agents of the plurality of mesh agents within the full mesh network or the partial mesh network to perform augmentations of events.
13. The method of claim 12 , wherein the augmentations are performed based on rule sets, and the use of the augmentation provisioning rules comprises provisioning the rule sets to the determined locations for the selected mesh agents of the plurality of mesh agents.
14. The method of claim 13 , further comprising routing of events within the distributed computing environment based on the rule sets.
15. The method of claim 13 , comprising creating configuration data for each of the plurality of computational nodes, the configuration data containing a specific rule set of the rule sets to be provisioned to a specific node.
16. The method of claim 1 , wherein applying the augmentation to the event comprises:
determining the augmentation is associated with the event producer and that the augmentation has no location restrictions; and
assigning the augmentation to a node of the plurality of computational nodes corresponding to a location of the event producer.
17. The method of claim 1 , wherein applying the augmentation to the event comprises:
determining the augmentation is associated with the event consumer and that the augmentation has no location restrictions; and
assigning the augmentation to a node of the plurality of computational nodes corresponding to a location of the event consumer.
18. The method of claim 1 , wherein applying the augmentation to the event comprises:
determining that the augmentation has a location restriction indicating a restriction on applying the augmentation; and
selecting a node along the low-cost route for applying the augmentation to the event based on determining that the node satisfies the restriction.
19. A system comprising:
one or more processors; and
a non-transitory computer readable storage medium comprising instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
storing transmission costs between each of a plurality of pairs of computational nodes of a plurality of computational nodes in a distributed computing environment, the plurality of computational nodes comprising event producers and event consumers;
determining a low-cost route to deliver an event detected at an event producer to an event consumer based on the transmission costs;
applying, via the low-cost route, an augmentation to the event to generate an augmented event; and
delivering, via the low-cost route, the augmented event to the event consumer.
20. A machine-readable non-transitory storage medium having instruction data executable by a machine to cause the machine to perform operations comprising:
storing transmission costs between each of a plurality of pairs of computational nodes of a plurality of computational nodes in a distributed computing environment, the plurality of computational nodes comprising event producers and event consumers;
determining a low-cost route to deliver an event detected at an event producer to an event consumer based on the transmission costs;
applying, via the low-cost route, an augmentation to the event to generate an augmented event; and
delivering, via the low-cost route, the augmented event to the event consumer.