Employing triggered retention in an ordered event stream storage system
Retention of events of an ordered event stream according to at least one triggered retention policy is disclosed. Expiration of events stored in a segment of an ordered event stream (OES) can be desirable. New events can be added to a head of an OES segment, and pruning events from a tail of the OES segment can be desirable. Processing applications can predicate event retention, e.g., restricting expiration of an event, on at least one triggered retention policy. In some embodiments, an additional fixed retention policy can be combined with the triggered retention. The disclosed retention can be performed at the event-level or at less granular levels, e.g., segment-level, OES-level, etc., e.g., via batching of events.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
determining a batch of events to be deleted from an ordered event stream stored via an ordered event stream storage system, wherein the batch of events corresponds to one or more triggered retention windows;
determining a first count of retention windows that have not elapsed, wherein the determining the first count of retention windows that have not elapsed comprises determining a second count of unelapsed triggered retention windows of the one or more triggered retention windows; and
in response to determining that the first count is zero, facilitating deletion of events comprised in the batch of events.
2. The system of claim 1 , wherein a triggered retention window of the one or more triggered retention windows is an affirmatively triggered retention window.
3. The system of claim 1 , wherein a triggered retention window of the one or more triggered retention windows is a negatively triggered retention window.
4. The system of claim 3 , wherein a progression of the negatively triggered retention window is restarted in response to an affirmative trigger occurring.
5. The system of claim 1 , wherein the batch of events corresponds to one or more fixed retention windows, and wherein the determining the first count of retention windows that have not elapsed comprises determining a third count of unelapsed fixed retention windows of the one or more fixed retention windows.
6. The system of claim 1 , wherein the determining the batch of events to be deleted is based on receiving a request to truncate the ordered event stream from a determined progress point.
7. The system of claim 1 , wherein the determining the batch of events to be deleted results in the batch comprising one event.
8. The system of claim 1 , wherein the determining the batch of events to be deleted results in the batch comprising more than one event.
9. The system of claim 8 , wherein the more than one event are stored in one segment of the ordered event stream.
10. The system of claim 8 , wherein the more than one event are stored among more than one segment of the ordered event stream.
11. A method, comprising:
in response to receiving a request to delete one or more events stored via an ordered event stream storage system, determining, by a system comprising a processor, a batch of events to be deleted from the ordered event stream storage system, wherein the batch of events corresponds to one or more triggered retention windows comprising a first count of retention windows that have not elapsed and a second count of triggered retention windows that have not elapsed;
determining, by the system, that at least one retention policy corresponding to the batch of events has been satisfied; and
indicating, by the system, a permission to delete the events of the batch of events based on the determining that the at least one retention policy has been satisfied.
12. The method of claim 11 , wherein the determining that at least one retention policy has been satisfied comprises determining that at least one triggered retention policy has been satisfied.
13. The method of claim 12 , wherein the determining that at least one retention policy has been satisfied further comprises determining that at least one fixed retention policy has been satisfied.
14. The method of claim 11 , wherein the one or more events are stored via one or more segments of one or more ordered event streams of one or more scopes of the ordered event stream storage system, and wherein a scope is equivalent to a namespace.
15. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
determining that a request to delete one or more events stored via an ordered event stream storage system has occurred;
selecting a batch of events to be deleted from the ordered event stream storage system based on the request, wherein the batch of events corresponds to one or more triggered retention windows comprising a first count of retention windows that have not elapsed and a second count of triggered retention windows that have not elapsed; and
in response to determining that at least one triggered retention policy corresponding to the batch of events has been satisfied, facilitating deletion of the events of the batch of events.
16. The non-transitory machine-readable medium of claim 15 , wherein the at least one triggered retention policies comprise an affirmatively triggered retention policy that activates a first triggered retention window in response to determining an occurrence of a trigger.
17. The non-transitory machine-readable medium of claim 15 , wherein the at least one triggered retention policies comprise an inverted trigger retention policy that is permits a first triggered retention window to be active in response to determining an occurrence of a trigger has not occurred.
18. The non-transitory machine-readable medium of claim 17 , wherein the inverted trigger retention policy resets elapsed activity of the first triggered retention window as a second triggered retention window in response to determining an occurrence of a trigger has occurred.
19. The non-transitory machine-readable medium of claim 15 , wherein the facilitating of the deletion of the events of the batch of events is further in response to determining that at least one fixed retention policy corresponding to the batch of events has also been satisfied.
20. The non-transitory machine-readable medium of claim 15 , wherein a triggered retention window of the one or more triggered retention windows is a negatively triggered retention window.