Scalable parallel messaging process
Methods and apparatus, including computer program products, implementing and using techniques for event processing in a content management system. In response to determining that an inter-dependence exists between two or more events in the content management system, a same group event key is assigned to each inter-dependent event. All events having the same group event key are distributed to a same event processing node on a consistent hashing ring.
1. A computer program product for inter-dependent event message processing in a content management system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions being executable executed by a processor to cause the processor to perform a method comprising:
receiving a first transactional event in the content management system, the first transactional event including a first set of attribute values, and one or more of: a first event identifier, a first event process status, and a first event message body;
receiving a second transactional event in the content management system, the second transactional event including a second set of attribute values, and one or more of: a second event identifier, a second event process status, and a second event message body;
in response to determining, by the processor and based on a set of pre-defined rules, that an inter-dependence exists between the first transactional event and the second transactional event, the inter-dependence requiring the first transactional event and the second transactional event to be processed in a sequential order in order to maintain data integrity of a transaction involving the first transactional event and the second transactional event, assigning, by the processor, a same event group key to the first transactional event and to the second transactional event, wherein the event group key is generated by a hashing function based on common attribute values in the first set of attribute values and the second set of attribute values;
distributing, by the processor, all transactional events having a same event group key to a same event processing node on a consistent hashing ring;
processing in parallel, by each event processing node on the consistent hashing ring, groups of inter-dependent transactional events; and
in response to detecting, by the processor, that a processing node is unavailable, removing, by the processor, the processing node from the consistent hashing ring and transferring, by the processor, transactional events assigned to the unavailable processing node from the unavailable processing node to a different processing node on the consistent hashing ring.
2. The computer program product of claim 1 , wherein the method further comprises:
storing, by the processor, the transactional events in an event table from which event processing nodes pull transactional events for processing.
3. The computer program product of claim 1 , wherein the method further comprises:
balancing, by the processor, the utilization of different processing nodes based on a workload for each processing node.
4. The computer program product of claim 1 , wherein the method further comprises:
in response to adding a processing node to the consistent hashing ring, remapping, by the processor, transactional events on processing nodes affected by the addition of the new processing node, while leaving processing nodes not affected by the new processing node unchanged.
5. A system for event message processing in a content management system, comprising:
a processor; and
a memory containing program instructions being executed by the processor to cause the processor to perform a method comprising:
receiving a first transactional event in the content management system, the first transactional event including a first set of attribute values, and one or more of: a first event identifier, a first event process status, and a first event message body;
receiving a second transactional event in the content management system, the second transactional event including a second set of attribute values, and one or more of: a second event identifier, a second event process status, and a second event message body;
in response to determining, by the processor and based on a set of pre-defined rules, that an inter-dependence exists between the first transactional event and the second transactional event, the inter-dependence requiring the first transactional event and the second transactional event to be processed in a sequential order in order to maintain data integrity of a transaction involving the first transactional event and the second transactional event, assigning, by the processor, a same event group key to the first transactional event and to the second transactional event, wherein the event group key is generated by a hashing function based on common attribute values in the first set of attribute values and the second set of attribute values;
distributing, by the processor, all transactional events having a same event group key to a same event processing node on a consistent hashing ring;
processing in parallel, by each event processing node on the consistent hashing ring, groups of inter-dependent transactional events; and
in response to detecting, by the processor, that a processing node is unavailable, removing, by the processor, the processing node from the consistent hashing ring and transferring, by the processor, transactional events assigned to the unavailable processing node from the unavailable processing node to a different processing node on the consistent hashing ring.