Scalable parallel messaging process
Methods for event processing in a content management system. In response to determining that an inter-dependence exists between two or more events in the content management system, a same group event key is assigned to each inter-dependent event. All events having the same group event key are distributed to a same event processing node on a consistent hashing ring.
1. A computer-implemented method for inter-dependent event processing in a content management system, comprising:
receiving a first transactional event in the content management system, the first transactional event including a first set of attribute values, and one or more of: a first event identifier, a first event process status, and a first event message body;
receiving a second transactional event in the content management system, the second transactional event including a second set of attribute values, and one or more of: a second event identifier, a second event process status, and a second event message body;
in response to determining, by the processor and based on a set of pre-defined rules, that an inter-dependence exists between the first transactional event and the second transactional event, the inter-dependence requiring the first transactional event and the second transactional event to be processed in a sequential order in order to maintain data integrity of a transaction involving the first transactional event and the second transactional event, assigning, by the processor, a same event group key to the first transactional event and to the second transactional event, wherein the event group key is generated by a hashing function based on common attribute values in the first set of attribute values and the second set of attribute values;
distributing, by the processor, all transactional events having a same event group key to a same event processing node on a consistent hashing ring;
processing in parallel, by each event processing node on the consistent hashing ring, groups of inter-dependent transactional events; and
in response to detecting, by the processor, that a processing node is unavailable, removing, by the processor, the processing node from the consistent hashing ring and transferring, by the processor, transactional events assigned to the unavailable processing node from the unavailable processing node to a different processing node on the consistent hashing ring.
2. The method of claim 1 , further comprising:
storing, by the processor, the transactional events in an event table from which event processing nodes pull transactional events for processing.
3. The method of claim 1 , further comprising:
balancing, by the processor, the utilization of different processing nodes based on a workload for each processing node.
4. The method of claim 1 , further comprising:
in response to adding a processing node to the consistent hashing ring, remapping, by the processor, transactional events on processing nodes affected by the addition of the new processing node, while leaving processing nodes unaffected by the new processing node unchanged.