Information-unit based scaling of an ordered event stream
Scaling an ordered event stream (OES) based on an information-unit (IU) metric is disclosed. The IU metric can correspond to an amount of computing resources that can be consumed to access information embodied in event data of an event of the OES. In this regard, the amount of computing resources to access the data of the stream event itself can be distinct from an amount of computing resources employed to access information embodied in the data. As such, where an external application, e.g., a reader, a writer, etc., can connect to an OES data storage system, enabling the OES to be scaled in response to burdening of computing resources accessing event information, rather than merely event data, can aid in preservation of an ordering of events accessed from the OES.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor perform operations comprising:
receiving a value indicating a predicted level of burden on computing resources of the system corresponding to a reader application instance reading information embodied in event data of an ordered event stream (OES) to be written to a storage device of the system, wherein the value is based on predicted performance data of a simulation of the reader application instance exceeding a threshold;
updating a topology of a key space of the OES, wherein the updating reduces the predicted level of burden on the computing resources, and wherein the updating is permitted based on a condition being satisfied;
scaling the OES based on the updated topology; and
writing the scaled OES to the storage device, wherein the information embodied in the event data of the OES is read by the reader application instance.
2. The system of claim 1 , wherein the scaling is a scaling up of the OES, and wherein the reader application instance is one of a plurality of reader application instances.
3. The system of claim 1 , wherein the scaling is a scaling down of the OES, and wherein the condition is satisfied when values for every reader application instance of a plurality of reader application instances satisfy a rule.
4. The system of claim 1 , wherein the reader application instance is in communication with, but is not executing on, a component comprised in the system.
5. The system of claim 1 , wherein the event data of the OES is received from a writer application instance.
6. The system of claim 5 , wherein the operations further comprise performing, in response to receiving a future event reading operation from the reader application instance, performing a supplemental scaling of the OES based on feedback from the reader application instance.
7. The system of claim 5 , wherein the writer application instance is in communication with, but is not executing on, a component comprised in the system.
8. The system of claim 1 , wherein a division of the OES into at least two segments is based on the updated topology.
9. The system of claim 1 , wherein a merging of the OES with another OES is based on the updated topology.
10. A method, comprising:
receiving, by a system comprising a processor, a normalized value indicating a predicted level of burden on computing resources of the system corresponding to a reader application instance reading information embodied in event data of an ordered event stream (OES) to be written in a storage device of the system, wherein the value is based on predicted performance data of a simulation of the reader application instance exceeding a threshold;
updating a key space topology corresponding to the OES, wherein the updating reduces the predicted level of burden on the computing resources, and wherein the updating is permitted based on a condition being satisfied;
scaling the OES based on the updated topology; and
writing the scaled OES to the storage device, wherein the information embodied in the event data of the OES is read by the reader application instance.
11. The method of claim 10 , wherein the normalized value is further based on receiving reader application instance feedback corresponding to the reader application instance reading the OES.
12. The method of claim 11 , wherein the reader application instance feedback is received from a reader application instance that is executed via another system.
13. The method of claim 10 , wherein the normalized value is further based on receiving, by the system, the event data from a writer application instance prior to writing the event data to the OES and a model of the reader application instance.
14. The method of claim 13 , wherein the event data is appended to the terminus of the OES.
15. The method of claim 13 , wherein the writer application instance is executed via a second system other than the system.
16. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor of a system perform operations comprising:
receiving, by the system, a normalized value indicating a predicted level of burden on computing resources of the system corresponding to a reader application instance reading information embodied in event data of an ordered event stream (OES) to be written to a storage device of the system, wherein the value is based on predicted performance data of a simulation of the reader application instance exceeding a threshold;
updating a topology for a key space of the OES, wherein the updating reduces the predicted level of burden on the computing resources, and wherein the updating is permitted based on a condition being satisfied;
scaling the OES based on the updated topology; and
writing the scaled OES to the storage device, wherein the information embodied in the event data of the OES is read by the reader application instance.
17. The non-transitory machine-readable medium of claim 16 , wherein the normalized value is further based on reader application instance feedback corresponding to the reader application instance reading the OES.
18. The non-transitory machine-readable medium of claim 17 , wherein the reader application instance feedback is received from a reader application instance that is executed via another system.
19. The non-transitory machine-readable medium of claim 16 , wherein the event data is received from a writer application instance.
20. The non-transitory machine-readable medium of claim 19 , wherein the writer application instance is executed via a second system other than the system.