Techniques for managing storage and retention of time-series based event data

1 . A method comprising:

determining historical query information based upon historical execution of a query over an event stream;

identifying a storage attribute of data associated with the historical execution of the query over the event stream, the storage attribute indicating at least one of a size of the data or an assigned storage location of the data in a first storage location or a second storage location, wherein the data is an event representation of a discoverable event stream, the event representation identifying an event of an object of the discoverable event stream;

identifying an execution attribute of a future execution of the query over the event stream including that the query is executed within an upcoming period of time;

modifying, based on the historical query information and based on the execution attribute indicating that the query is executed within the upcoming period of time, the assigned storage location of the data from a first level of a multi-tiered storage system to assign the data to non-persistent cache memory in a second level of the multi-tiered storage system;

storing, based on modifying the assigned storage location of the data, the data in the non-persistent cache memory;

identifying one or more other event representations associated with the object;

determining a query attribute of a future execution of the query over the discoverable event stream;

determining whether to delete the event representation based upon the historical query information, the one or more other event representations, and/or the query attribute; and

updating the data based on the determining whether to delete the event representation.

2 . The method of claim 1 , wherein the execution attribute is a first execution attribute, and further comprising:

identifying, for deletion from the event stream based on an age of the data or a position of the data within the event stream, query data associated within the event stream; and

determining, based on a second execution attribute, whether to delete the data from the event stream.

3 . The method of claim 1 , wherein one of the first level or the second level is a persistent storage, and the other one of the first level or the second level is a non-persistent storage.

4 . The method of claim 1 , wherein the execution attribute is a next execution of the query, a look back threshold of the query, or a tolerance delay of the query.

5 . The method of claim 1 , wherein

determining whether to delete the event representation is further based upon an age of the event representation and a retention policy for retaining data in the discoverable event stream.

6 . The method of claim 5 , wherein the retention policy indicates to delete the event representation based on being an oldest among a plurality of event representations.

7 . The method of claim 1 , wherein determining whether to delete the event representation comprises retaining the event representation based upon the event representation being associated with a query scheduled to execute within a predefined period.

8 . The method of claim 1 , wherein determining whether to delete the event representation comprises deleting the event representation based upon the event representation being associated with a query that is not scheduled to execute within a predefined period.

9 . An apparatus, comprising:

a memory; and

a processor coupled with the memory and configured to:

determine historical query information based upon historical execution of a query over an event stream;

identify a storage attribute of data associated with the historical execution of the query over the event stream, the storage attribute indicating at least one of a size of the data or an assigned storage location of the data in a first storage location or a second storage location, wherein the data is an event representation of a discoverable event stream, the event representation identifying an event of an object of the discoverable event stream;

identify an execution attribute of a future execution of the query over the event stream including that the query is executed within an upcoming period of time;

modify, based on the historical query information and based on the execution attribute indicating that the query is executed within the upcoming period of time, the assigned storage location of the data from a first level of a multi-tiered storage system to assign the data to non-persistent cache memory in a second level of the multi-tiered storage system;

store, based on the modifying the assigned storage location of the data, the data in the non-persistent cache memory;

identify one or more other event representations associated with the object;

determine a query attribute of a future execution of the query over the discoverable event stream;

determine whether to delete the event representation based upon the historical query information, the one or more other event representations, and/or the query attribute; and

update the data based on the determining whether to delete the event representation.

10 . The apparatus of claim 9 , wherein the execution attribute is a first execution attribute, and wherein the processor is further configured to:

identify, for deletion from the event stream based on an age of the data or a position of the data within the event stream, query data associated within the event stream; and

determine, based on a second execution attribute, whether to delete the data from the event stream.

11 . The apparatus of claim 9 , wherein the first storage location is a persistent storage, and the second storage location is a non-persistent storage, or the first storage location is a first level of persistent storage or non-persistent storage, and the second storage location is a second level of persistent storage or non-persistent storage.

12 . The apparatus of claim 9 , wherein the execution attribute is a next execution of the query, a look back threshold of the query, or a tolerance delay of the query.

13 . The apparatus of claim 9 , wherein the data is an event representation of a discoverable event stream, the event representation identifying an event of an object of the discoverable event stream, and the processor is further configured to:

identify a storage attribute of data associated with the historical execution of the query over the discoverable event stream;

determine, based on the historical query information and/or the storage attribute, that a query performance attribute is greater than a predefined threshold; and

repartition, dynamically, a cluster resource based at least in part on the query performance attribute being greater than the predefined threshold, the repartitioning modifying an assignment of a plurality of objects of the discoverable event stream to the cluster resource.

14 . The apparatus of claim 9 , wherein the processor is further configured to

determine whether to delete the event representation based upon an age of the event representation and a retention policy for retaining data in the discoverable event stream.

15 . The apparatus of claim 9 , wherein to determine whether to delete the event representation, the processor is further configured retain the event representation based upon the event representation being associated with a query scheduled to execute within a predefined period.

16 . The apparatus of claim 9 , wherein to determine whether to delete the event representation, the processor is further configured delete the event representation based upon the event representation being associated with a query that is not scheduled to execute within a predefined period.

17 . A non-transitory computer-readable device having instructions thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:

determining historical query information based upon historical execution of a query over an event stream;

identifying a storage attribute of data associated with the historical execution of the query over the event stream, the storage attribute indicating at least one of a size of the data or an assigned storage location of the data in a first storage location or a second storage location, wherein the data is an event representation of a discoverable event stream, the event representation identifying an event of an object of the discoverable event stream;

identifying an execution attribute of a future execution of the query over the event stream including that the query is executed within an upcoming period of time;

modifying, based on the historical query information and based on the execution attribute indicating that the query is executed within the upcoming period of time, the assigned storage location of the data from a first level of a multi-tiered storage system to assign the data to non-persistent cache memory in a second level of the multi-tiered storage system;

storing, based on modifying the assigned storage location of the data, the data in the non-persistent cache memory;

identifying one or more other event representations associated with the object;

determining a query attribute of a future execution of the query over the discoverable event stream;

determining whether to delete the event representation based upon the historical query information, the one or more other event representations, and/or the query attribute; and

updating the data based on the determining whether to delete the event representation.

18 . The non-transitory computer-readable device of claim 17 , wherein

determining whether to delete the event representation is further based upon an age of the event representation and a retention policy for retaining data in the discoverable event stream.