Conditional appends in an ordered event stream storage system
Conditionally appending an event in an ordered event stream (OES) is disclosed. An OES storage system can comprise an OES that can have a dynamic topology, e.g., comprising changing numbers of segments, segments that have changing routing key ranges, mixes of local and remote storage devices and processors, employing different types of storage, or combinations thereof. A conditional append operation can be predicated on determining an appropriate segment, for example, based on a routing key. Conditions of the conditional appends can be designated by a user. Failing conditions can trigger a corrective action, throwing of an error. Satisfied conditions can result in permitting the conditional append. Moreover, the conditional append can be dependent on populating a group of attributes for the correct segment based on attributes that can be determined for a different epoch of the OES, again for example, based on a routing key.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
receiving an append operation corresponding to writing an event to a segment of an ordered event stream;
in response to determining that the append operation is an unconditional append operation, enabling writing of the event to the segment of the ordered event stream;
in response to determining that the append operation is a composite unconditional append operation based on the append operation comprising a group of attribute updates, wherein the group of attribute updates does not indicate a condition of conditions to be satisfied before permitting writing of the event to the ordered event stream, enabling writing of the event to the segment of the ordered event stream and enabling updating of a stream attribute; and
in response to determining that the append operation is a composite conditional append operation, determining a condition state based on the stream attribute and an attribute update;
in response to the condition state indicating satisfaction of the conditions, enabling writing of the event to the segment of the ordered event stream and enabling updating of stream attributes; and
in response to the condition state indicating that at least one condition of the conditions is not satisfied, prohibiting writing of the event to the segment of the ordered event stream.
2. The system of claim 1 , wherein the determining that the append operation is an unconditional append operation is based on the append operation not comprising a group of attribute updates.
3. The system of claim 1 , wherein the determining that the append operation is a composite conditional append operation is based on the append operation comprising a group of attribute updates, and wherein the group of attribute updates indicates a condition of the conditions to be satisfied before permitting writing of the event to the ordered event stream.
4. The system of claim 1 , wherein the append operation indicates a routing key corresponding to the event, and wherein the segment of the ordered event stream is determined from the routing key.
5. The system of claim 1 , wherein the determining the condition state comprises accessing a first segment attribute corresponding to the segment in a first epoch of the ordered event stream.
6. The system of claim 5 , wherein the determining the condition state further comprises accessing a second segment attribute corresponding to another segment in a second epoch of the ordered event stream, wherein the second epoch is a different epoch than the first epoch, wherein the first segment attribute corresponds to a routing key, wherein the second segment attribute also corresponds to the routing key, and wherein the second segment attribute is substituted for the first segment attribute.
7. The system of claim 6 , wherein the ordered event stream is scalable, enabling a change in a topology of the ordered event stream.
8. The system of claim 7 , wherein the change in a topology of the ordered event stream results in a routing key affiliated with a first segment in a first epoch of the ordered event stream being affiliated with a second segment in a second epoch of the ordered event stream, and wherein the first segment is a different segment than the second segment.
9. A method, comprising:
receiving, by a system comprising a processor, an append operation corresponding to appending an event to a segment of an ordered event stream, wherein the append operation indicates an event and a routing key that corresponds to the event;
in response to determining that the append operation is an unconditional append operation, permitting, by the system, appending of the event to the segment of the ordered event stream;
in response to determining that the append operation is a composite unconditional append operation based on the append operation comprising a group of attribute updates, wherein the group of attribute updates does not indicate a condition of conditions to be satisfied before permitting writing of the event to the ordered event stream, permitting, by the system, appending of the event to the segment of the ordered event stream; and
in response to determining that the append operation is a composite conditional append operation, determining, by the system, a condition state based on a stream attribute and an attribute update of the attribute updates;
in response to the condition state indicating that conditions have been satisfied, permitting, by the system, appending of the event to the segment of the ordered event stream; and
in response to the condition state indicating that at least one of the conditions is not satisfied, prohibiting, by the system, appending of the event to the segment of the ordered event stream.
10. The method of claim 9 , wherein the determining that the append operation is an unconditional append operation comprises determining that the append operation does not comprise a group of attribute updates.
11. The method of claim 9 , wherein the determining that the append operation is a composite conditional append operation comprises determining the append operation indicates a group of attribute updates that indicate at least one condition to be satisfied before permitting appending of the event to the ordered event stream.
12. The method of claim 9 , further comprising determining, by the system, the segment of the ordered event stream based on the routing key.
13. The method of claim 9 , wherein the determining the condition state comprises a first seeking, by the system, of a first segment attribute corresponding to the segment in a first epoch of the ordered event stream, and a second seeking of a second segment attribute corresponding to another segment in a second epoch of the ordered event stream, wherein the second epoch is a different epoch than the first epoch, wherein the first segment attribute corresponds to the routing key, wherein the second segment attribute also corresponds to the routing key, and wherein the second segment attribute is substituted for the first segment attribute in response to the first segment attribute not being fetched as a result of the first seeking and the second segment attribute being fetched as a result of the second seeking.
14. The method of claim 9 , wherein the operations further comprise supporting, by the system, a change in a topology of the ordered event stream where the ordered event stream is determined to be scalable.
15. The method of claim 14 , wherein the change in the topology of the ordered event stream comprises affiliating, by the system, a routing key of a first segment in a first epoch of the ordered event stream being with a second segment in a second epoch of the ordered event stream.
16. The method of claim 14 , wherein the first segment is a different segment than the second segment.
17. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
in response to receiving an append operation corresponding to appending an event to a segment of an ordered event stream, determining a type of the append operation, wherein the append operation indicates an event and a routing key;
where the type of the append operation is an unconditional append operation type, allowing writing of the event to the segment of the ordered event stream;
where the type of the append operation is a composite unconditional append operation type, allowing writing of the event to the segment of the ordered event stream based on the append operation comprising a group of attribute updates, wherein the group of attribute updates does not indicate a condition of conditions to be satisfied before permitting writing of the event to the ordered event stream; and
where the type of the append operation is a composite conditional append operation type, determining a condition state based on a stream attribute and an attribute update of the attribute updates;
where the condition state corresponds to a group of defined conditions having been satisfied, allowing writing of the event to the segment of the ordered event stream; and
where the condition state corresponds to at least one of the group of conditions not having been satisfied, prohibiting writing of the event to the segment of the ordered event stream.
18. The non-transitory machine-readable medium of claim 17 , wherein the determining that the append operation is an unconditional append operation comprises determining that the append operation does not comprise a group of attribute updates.
19. The non-transitory machine-readable medium of claim 17 , wherein the determining that the append operation is a composite conditional append operation comprises determining the append operation indicates a group of attribute updates indicating at least one condition to be satisfied as a prerequisite to appending of the event to the ordered event stream.
20. The non-transitory machine-readable medium of claim 17 , wherein the determining the condition state comprises a first attempt to access a first segment attribute corresponding to the segment in a first epoch of the ordered event stream, and a second attempt to access a second segment attribute corresponding to another segment in a second epoch of the ordered event stream, wherein the second epoch is a different epoch than the first epoch, wherein the first segment attribute corresponds to the routing key, wherein the second segment attribute also corresponds to the routing key, and wherein the second segment attribute is substituted for the first segment attribute in response to the first segment attribute not being fetched as a result of the first attempt to access and the second segment attribute being fetched as a result of the second attempt to access.