Method, system and apparatus for dynamically allocating event data

1. A method of dynamically allocating event data from a plurality of client devices among a set of event processors, the method comprising:

at a partitioning controller, storing an initial shard map allocating initial subsets of the client devices to respective data stores, each data store associated with a respective one of the event processors;

at the partitioning controller, obtaining an operational parameter for each of the event processors;

at the partitioning controller, generating an updated shard map based on the operational parameter for each of the event processors, the updated map allocating updated subsets of the client devices to the respective data stores;

responsive to generating the updated shard map, transmitting a map update notification from the partitioning controller for the client devices;

receiving, at a first one of the event processors, a reallocation initiation message from a reallocated one of the client devices; and

when no event data from the reallocated client device remains to be processed at the first event processor, sending a reallocation completion message to a second event processor.

2. The method of claim 1 , wherein the initial shard map defines a plurality of client identifier ranges, and an identifier of a respective event processor corresponding to each client identifier range.

3. The method of claim 1 , wherein obtaining an operational parameter for each of the event processors comprises:

transmitting an operational parameter request to each event processor; and

receiving respective operational parameters from each event processor.

4. The method of claim 1 , wherein each operational parameter indicates a workload at the corresponding event processor.

5. The method of claim 4 , wherein each operational parameter includes at least one of central processing unit (CPU) utilization, a message processing rate, and a message queue length.

6. The method of claim 4 , wherein generating the updated shard map comprises determining whether the workloads exceed a predetermined threshold.

7. The method of claim 6 , wherein determining whether the workloads exceed a predetermined threshold comprises:

determining whether any of the workloads exceed a first threshold; and

determining whether a difference between a maximum one of the workloads and a minimum one of the workloads exceeds a second threshold.

8. The method of claim 1 , further comprising:

prior to receiving the reallocation completion message at the second event processor, receiving further event data from the reallocated client device; and

storing the further event data in a buffer pending receipt of the reallocation completion message.

9. A system comprising:

a plurality of client devices generating event data;

a plurality of event processors, each event processor configured to process the event data from an allocated one or more of the plurality of client devices, the plurality of event processors including a first one of the event processors and a second one of the event processors, the first event processor configured to:

receive a reallocation initiation message from a reallocated one of the plurality of client devices; and

when no event data from the reallocated client device remains to be processed at the first event processor, send a reallocation completion message to a second event processor; and

a partitioning controller including (i) a communications interface, (ii) a memory storing an initial shard map allocating initial subsets of the plurality of client devices to respective data stores, each data store associated with a respective one of the plurality of event processors, and (iii) a processor connected with the communications interface and the memory, the processor configured to:

obtain an operational parameter for each of the plurality of event processors;

generate an updated shard map based on the operational parameter for each of the plurality of event processors, the updated map allocating updated subsets of the plurality of client devices to the respective data stores; and

responsive to generating the updated shard map, transmit a map update notification for the plurality of client devices via the communications interface.

10. The system of claim 9 , wherein the initial shard map defines a plurality of client identifier ranges, and an identifier of a respective event processor corresponding to each client identifier range.

11. The system of claim 9 wherein, to obtain an operational parameter for each of the plurality of event processors, the processor is further configured to:

transmit an operational parameter request to each event processor via the communications interface; and

receive, via the communications interface, respective operational parameters from each event processor.

12. The system of claim 9 , wherein each operational parameter indicates a workload at the corresponding event processor.

13. The system of claim 12 , wherein each operational parameter includes at least one of central processing unit (CPU) utilization, a message processing rate, and a message queue length.

14. The system of claim 12 , wherein generating the updated shard map comprises determining whether the workloads exceed a predetermined threshold.

15. The system of claim 14 wherein, to determine whether the workloads exceed a predetermined threshold, the processor is further configured to:

determine whether any of the workloads exceed a first threshold; and

determine whether a difference between a maximum one of the workloads and a minimum one of the workloads exceeds a second threshold.

16. The system of claim 9 , wherein the second event processor is configured to:

prior to receiving the reallocation completion message from the first event processor, receive further event data from the reallocated client device; and

store the further event data in a buffer at the second event processor pending receipt of the reallocation completion message.