Dynamically instantiating and terminating data queues

1. A computer-implemented method comprising:

receiving live data at a data queue system (DQS), wherein the live data is associated with an entity;

dynamically instantiating a dual-queue node associated with the entity on the DQS, wherein dynamically instantiating the dual-queue node includes:

initializing a live data queue included in the dual-queue node, wherein the initialized live data queue is enabled to receive and process the live data and wherein the live data queue is assigned to the entity; and

determining whether a persistent data store of the DQS includes a partition assigned to the entity, wherein the partition assigned to the entity is enabled to serve as a stale data queue for the dual-queue node, and wherein the stale data queue stores a persistent copy of the live data;

routing the live data to the dual-queue node; and

employing the dual-queue node to process the live data.

2. The computer-implemented method of claim 1 , further comprising:

determining whether the DQS currently includes the dual-queue node associated with the entity; and

instantiating the dual queue node associated with the entity based on determining whether the DQS currently includes the dual-queue node associated with the entity.

3. The computer-implemented method of claim 1 , further comprising:

receiving additional live data at the DQS, wherein the additional live data is associated with the entity;

in response to determining that the DQS includes the dual-queue node associated with the entity, routing the additional live data to the dual-queue node; and

employing the dual-queue data node to process the additional live data.

4. The computer-implemented method of claim 1 , further comprising:

determining whether the dual-queue node is active based on a period of inactivity; and

in response to determining that the dual-queue node is active, employing the active dual-queue node to process the additional live data.

5. The computer-implemented method of claim 1 , further comprising:

determining whether the dual-queue node is inactive based on a period of inactivity; and

in response to determining that the dual-queue node is inactive, terminating the dual-queue node.

6. The computer-implemented method of claim 1 , further comprising:

in response to determining that the dual-queue node is inactive, deallocating memory associated with the live data queue.

7. The computer-implemented method of claim 1 , further comprising:

in response to determining that the dual-queue node is inactive, preserving stale data or state data stored in the stale data queue for the dual-queue node.

8. The computer-implemented method of claim 1 , further comprising:

in response to determining that the dual-queue node is inactive, terminating the dual-queue node; and

in response to receiving additional live data associated with the entity, dynamically re-instantiating the terminated dual-queue node based on preserved state data stored in the stale data queue for the terminated dual-queue node.

9. The computer-implemented method of claim 1 , further comprising:

determining whether to terminate the dual-queue node based on a period of inactivity, wherein the period of inactivity is dynamically adjusted based on an amount of data stored in the stale data queue of the dual-queue node, an activity level of the DQS, or an amount of available resources of the DQS.

10. The computer-implemented method of claim 1 , further comprising:

determining whether to terminate the dual-queue node based on a period of inactivity, wherein the period of inactivity is dynamically adjusted based on a determined re-instantiation period associated with the entity.

11. The computer-implemented method of claim 1 , further comprising:

in response to determining that the dual-queue node is inactive, updating queue-entity mapping data to remove the terminated dual-queue data node.

12. The computer-implemented method of claim 1 , further comprising:

determining whether the dual-queue node is inactive;

in response to determining that the dual-queue node is inactive, adding the dual-queue node to a prioritized queue of inactive dual-queue nodes; and

in response to determining that a utilization of resources of the DQS exceeds a predefined threshold, selecting at least one dual-queue node included in the prioritized queues for termination based on a prioritization associated with each of the inactive dual-queue nodes.

13. The computer-implemented method of claim 1 , further comprising:

initializing a dual-queue manager for the dual-queue node;

determining an amount of memory for the live data queue based on an activity level of the entity or an amount of available resources of the DQS; and

employing the dual-queue manager to allocate the determined amount of memory for the live data queue.

14. The computer-implemented method of claim 1 , wherein dynamically instantiating the dual-queue node associated with the entity further includes:

identifying the partition included in the persistent data store that is associated with the entity; and

associating the partition with the dual-queue node, such that the partition serves as the stale data queue of the dual-queue node.

15. The computer-implemented method of claim 1 , wherein dynamically instantiating the dual-queue node associated with the entity further includes:

in response to determining that the persistent data store does not include the partition associated with the entity, initializing the partition such that the persistent data store includes the partition; and

associating the partition with the dual-queue node, such that the partition serves as the stale data queue of the dual-queue node.

16. The computer-implemented method of claim 1 , wherein dynamically instantiating the dual-queue node associated with the entity further includes:

updating queue-entity mapping data to include the dual-queue node and the dual-queue node's association with the entity.

17. The computer-implemented method of claim 1 , wherein the DQS is a multi-tenant DQS and the method further comprises:

receiving a request to migrate the received live data to a single-tenant DQS associated with the entity;

dynamically instantiating another dual-queue node associated with the entity on the single-tenant DQS;

forwarding the live data to the other dual-queue node included in the single-tenant DQS; and

terminating the dual-queue node associated with the entity included in the multi-tenant DQS.

18. The computer-implemented method of claim 1 , wherein the method further comprises:

receiving a request to migrate the received live data to a single-tenant DQS associated with the entity;

dynamically instantiating another dual-queue node associated with the entity on the single-tenant DQS; and

forwarding a portion of the stale data stored in the stale data queue for the dual-queue node included in the multi-tenant DQS to the other dual-queue node included in the single-tenant DQS.

19. The computer-implemented method of claim 1 , wherein the method further comprises:

receiving a request for the live data associated with the entity;

determining that a copy of the live data is stored as stale data in the stale data queue for the dual-queue node;

retrieving the copy of the live data stored as stale data; and

employing the copy of the live data to service the request for the live data.

20. The computer-implemented method of claim 1 , wherein the method further comprises:

loading the live data into the live data queue of the dual-queue node; and

storing a persistent copy of the live data in the stale data queue of the dual-queue node.

21. The computer-implemented method of claim 1 , wherein the method further comprises:

establishing a connection with a remote data store;

employing the connection with the remote data store to transmit a persistent copy of the live data stored in the stale data queue of the dual-queue node to the remote data store; and

deleting the portion of the persistent copy of the live data from the stale data queue of the dual-queue node.

22. The computer-implemented method of claim 1 , wherein the method further comprises:

monitoring the stale data queue of the dual-queue node; and

in response to detecting a deletion event when monitoring the stale data queue of the dual-queue node, deleting a portion of stale data stored in the stale data queue for the dual-queue node.

23. The computer-implemented method of claim 1 , wherein processing the live data includes generating summary statistics associated with the entity and the method further comprises:

in response to generating the summary statistics, providing the summary statistics to the entity.

24. The computer-implemented method of claim 1 , wherein processing the live data includes detecting an alert event associated with the entity and the method further comprises:

in response to detecting the alert event, providing an alert to the entity.

25. A system comprising:

one or more processors; and

one or more computer-readable storage media containing instructions which, in response to execution by the one or more processors, cause the one or more processors to:

receive live data at a data queue system (DQS), wherein the live data is associated with an entity;

dynamically instantiate a dual-queue node associated with the entity on the DQS, wherein dynamically instantiating the dual-queue node includes:

initializing a live data queue included in the dual-queue node, wherein the initialized live data queue is enabled to receive and process the live data and wherein the live data queue is assigned to the entity; and

determining whether a persistent data store of the DQS includes a partition assigned to the entity, wherein the partition assigned to the entity is enabled to serve as a stale data queue for the dual-queue node, and wherein the stale data queue stores a persistent copy of the live data;

routing the live data to the dual-queue node; and

employing the dual-queue node to process the live data.

26. The system of claim 25 , wherein in response to execution of the instruction by the one or more processors, the one or more processors further:

determine whether the DQS currently includes the dual-queue node associated with the entity; and

instantiate the dual queue node associated with the entity based on determining whether the DQS currently includes the dual-queue node associated with the entity.

27. The system of claim 25 , wherein in response to execution of the instruction by the one or more processors, the one or more processors further:

receive additional live data at the DQS, wherein the additional live data is associated with the entity;

in response to determining that the DQS includes the dual-queue node associated with the entity, route the additional live data to the dual-queue node; and

employ the dual-queue data node to process the additional live data.

28. One or more non-transitory computer-storage media having executable instructions, which, when executed by a computing device, cause the computing device to:

receive live data at a data queue system (DQS), wherein the live data is associated with an entity;

dynamically instantiate a dual-queue node associated with the entity on the DQS, wherein dynamically instantiating the dual-queue node includes:

initializing a live data queue included in the dual-queue node, wherein the initialized live data queue is enabled to receive and process the live data and wherein the live data queue is assigned to the entity; and

determining whether a persistent data store of the DQS includes a partition assigned to the entity, wherein the partition assigned to the entity is enabled to serve as a stale data queue for the dual-queue node, and wherein the stale data queue stores a persistent copy of the live data;

route the live data to the dual-queue node; and

employ the dual-queue node to process the live data.

29. The one or more non-transitory computer-storage media of claim 28 , wherein when the instructions are executed by the computing device, cause the computing device to further:

determine whether the dual-queue node is active based on a period of inactivity; and

in response to determining that the dual-queue node is active, employing the active dual-queue node to process the additional live data.

30. The one or more non-transitory computer-storage media of claim 28 , wherein when the instructions are executed by the computing device, cause the computing device to further:

determining whether the dual-queue node is inactive based on a period of inactivity; and

in response to determining that the dual-queue node is inactive, terminating the dual-queue node.