Multi-cohort replay storage and retrospective anomaly detection analysis
Facilitating analytics such as retrospective analysis includes ingesting session data. It further includes summarizing the session data and generating leaf-level summarizations of leaf groups present in the session data. It further includes storing the leaf-level summarizations. It further includes deriving parent-level summarizations for a parent group from leaf-level summarizations of leaf groups that are children of the parent group.
1. A system, comprising:
one or more processors configured to:
temporarily store, in a session-level epoch store, ingested telemetry data pertaining to a plurality of individual sessions for a time-step, wherein each session is annotated with a combination of attribute values;
generate a leaf-level epoch store for the time-step, wherein generating the leaf-level epoch store for the time-step comprises generating leaf-level summarizations for leaf groups present for the time-step in the session-level epoch store, and wherein generating a leaf-level summarization for a leaf group comprises:
grouping together sessions in the session-level epoch store that share a same combination of attribute values into the leaf group; and
aggregating a portion of the telemetry data pertaining to the sessions grouped together into the leaf group that share the same combination of attribute values;
flush the session-level epoch store of the telemetry data pertaining to the plurality of individual sessions for the time-step, wherein ingested session-level telemetry data pertaining to a next time-step is subsequently stored in the session-level epoch store; and
retain the leaf-level epoch store generated for the time-step; and
a memory coupled to the one or more processors and configured to provide the one or more processors with instructions.
2. The system recited in claim 1 , wherein the leaf-level epoch store resides in-memory on a single compute node.
3. The system recited in claim 1 , wherein a parent-level summarization of telemetry data for a parent group of sessions is derivable from leaf-level summarizations for leaf groups that are children of the parent group.
4. The system recited in claim 1 , wherein generating the leaf-level summarization comprises determining a decomposable statistic based at least in part on the portion of the telemetry data pertaining to the sessions grouped together into the leaf group.
5. The system recited in claim 1 , wherein the leaf-level epoch store is retained for subsequent accessing during a data retrieval stage to determine input to provide to a timeseries analysis task.
6. A method, comprising:
temporarily storing, in a session-level epoch store, ingested telemetry data pertaining to a plurality of individual sessions for a time-step, wherein each session is annotated with a combination of attribute values;
generating a leaf-level epoch store for the time-step, wherein generating the leaf-level epoch store for the time-step comprises generating leaf-level summarizations for leaf groups present for the time-step in the session-level epoch store, and wherein generating a leaf-level summarization for a leaf group comprises:
grouping together sessions in the session-level epoch store that share a same combination of attribute values into the leaf group; and
aggregating a portion of the telemetry data pertaining to the sessions grouped together into the leaf group that share the same combination of attribute values;
flushing the session-level epoch store of the telemetry data pertaining to the plurality of individual sessions for the time-step, wherein ingested session-level telemetry data pertaining to a next time-step is subsequently stored in the session-level epoch store; and
retaining the leaf-level epoch store generated for the time-step.
7. The method of claim 6 , wherein the leaf-level epoch store resides in-memory on a single compute node.
8. The method of claim 6 , wherein a parent-level summarization of telemetry data for a parent group of sessions is derivable from leaf-level summarizations for leaf groups that are children of the parent group.
9. The method of claim 6 , wherein generating the leaf-level summarization comprises determining a decomposable statistic based at least in part on the portion of the telemetry data pertaining to the sessions grouped together into the leaf group.
10. The method of claim 6 , wherein the leaf-level epoch store is retained for subsequent accessing during a data retrieval stage to determine input to provide to a timeseries analysis task.
11. A system, comprising:
one or more processors configured to:
access a retained leaf-level epoch store that was previously generated for a time-step, wherein the leaf-level epoch store comprises leaf-level summarizations generated for leaf groups present for the time-step, and wherein a leaf-level summarization for a leaf group was generated at least in part by:
grouping together sessions in a session-level epoch store that share a same combination of attribute values into the leaf group, wherein ingested telemetry data pertaining to a plurality of individual sessions for the time-step was temporarily stored in the session-level epoch store, and wherein each session was annotated with a combination of attribute values; and
aggregating a portion of the telemetry data pertaining to the sessions grouped together into the leaf group that share the same combination of attribute values, wherein the session-level epoch store was flushed of the telemetry data pertaining to the plurality of individual sessions for the time-step, and wherein ingested session-level telemetry data pertaining to a next time-step was subsequently stored in the session-level epoch store;
derive, from leaf-level summarizations for at least one leaf group, a parent-level summarization of telemetry data for sessions comprised in a parent group of the at least one leaf group and that are associated with the time-step; and
provide the parent-level summarization of telemetry data as input to a timeseries analysis task; and
a memory coupled to the one or more processors and configured to provide the one or more processors with instructions.
12. The system recited in claim 11 , wherein the leaf-level epoch store resides in-memory on a single compute node.
13. The system recited in claim 11 , wherein the deriving is performed using a CUBE operation.
14. The system recited in claim 11 , wherein the timeseries analysis task comprises retrospective analysis.
15. A method, comprising:
accessing a retained leaf-level epoch store that was previously generated for a time-step, wherein the leaf-level epoch store comprises leaf-level summarizations generated for leaf groups present for the time-step, and wherein a leaf-level summarization for a leaf group was generated at least in part by:
grouping together sessions in a session-level epoch store that share a same combination of attribute values into the leaf group, wherein ingested telemetry data pertaining to a plurality of individual sessions for the time-step was temporarily stored in the session-level epoch store, and wherein each session was annotated with a combination of attribute values; and
aggregating a portion of the telemetry data pertaining to the sessions grouped together into the leaf group that share the same combination of attribute values, wherein the session-level epoch store was flushed of the telemetry data pertaining to the plurality of individual sessions for the time-step, and wherein ingested session-level telemetry data pertaining to a next time-step was subsequently stored in the session-level epoch store;
deriving, from leaf-level summarizations for at least one leaf group, a parent-level summarization of telemetry data for sessions comprised in a parent group of the at least one leaf group and that are associated with the time-step; and
providing the parent-level summarization of telemetry data as input to a timeseries analysis task.
16. The method of claim 15 , wherein the leaf-level epoch store resides in-memory on a single compute node.
17. The method of claim 15 , wherein the deriving is performed using a CUBE operation.
18. The method of claim 15 , wherein the timeseries analysis task comprises retrospective analysis.