Query-time data sessionization and analysis
A system and method for implementing an iterative query mechanism to facilitate query-time sessionization and analysis of data is disclosed. At least, the method includes determining a table of independent events, determining a query, the query including a parameter specifying a size limit on a sample of sessions, executing the query against the table of independent events, at the time of query execution, processing the table of independent events to reconstruct the sample of sessions approaching the size limit, and at the time of query execution, analyzing the sample of reconstructed sessions to generate a result.
1. A computer-implemented method comprising:
determining a table of independent events based on a stream of incoming, up-to-date, and raw event-based data, a plurality of data segments comprising the table of independent events being distributed across a cluster of computing devices;
determining a query, the query including a criteria for grouping a plurality of independent events from the table of independent events into a session and a parameter for specifying a size limit on reconstructing a sample of sessions from the table of independent events;
distributing the query in parallel to the cluster of computing devices for executing the query against the table of independent events;
at the time of query execution on each computing device in the cluster of computing devices, processing, in real time, the plurality of data segments comprising the table of independent events to dynamically and randomly reconstruct the sample of sessions using a random sampling method, the sample of sessions matching the criteria and the size limit in the query, a sampling rate for dynamically and randomly reconstructing the sample of sessions being a function of the size limit and the plurality of data segments comprising the table of independent events, the sample of sessions being representative of a whole set of possible sessions from the table of independent events, each session in the sample of sessions including a time-ordered series of independent events occurring within a period of time and mapped to a unique identifier; and
at the time of query execution on each computing device in the cluster of computing devices, analyzing, in real time, the sample of sessions to generate a result.
2. The computer-implemented method of claim 1 , further comprising:
at the time of query execution, processing the table of independent events to aggregate a plurality of independent events into the time-ordered series of independent events and reconstruct the sample of sessions based on the time-ordered series of independent events;
transforming the sample of sessions into a data structure; and
rendering a visualization based on the data structure.
3. The computer-implemented method of claim 2 , wherein determining the query further comprises:
receiving a user interaction in association with the visualization; and
determining the query based on the user interaction.
4. The computer-implemented method of claim 1 , further comprising performing funnel analysis using the result.
5. The computer-implemented method of claim 1 , wherein the criteria includes at least one from a timeframe, a set of users, and a geographical location.
6. The computer-implemented method of claim 1 , wherein the unique identifier includes at least one from a session identifier, client device identifier, and a user identifier.
7. The computer-implemented method of claim 1 , wherein the table of independent events is loaded with data retrieved from one of a streaming data source and a batch data source.
8. A system comprising:
one or more processors; and
a memory, the memory storing instructions, which when executed cause the one or more processors to:
determine a table of independent events based on a stream of incoming, up-to-date, and raw event-based data, a plurality of data segments comprising the table of independent events being distributed across a cluster of computing devices;
determine a query, the query including a criteria for grouping a plurality of independent events from the table of independent events into a session and a parameter for specifying a size limit on reconstructing a sample of sessions from the table of independent claims;
distributing the query in parallel to the cluster of computing devices to execute the query against the table of independent events;
at the time of query execution on each computing device in the cluster of computing devices, process, in real time, the plurality of data segments comprising the table of independent events to dynamically and randomly reconstruct the sample of sessions using a random sampling method, the sample of sessions matching the criteria and the size limit in the query, a sampling rate for dynamically and randomly reconstructing the sample of sessions being a function of the size limit and the plurality of data segments comprising the table of independent events, the sample of sessions being representative of a whole set of possible sessions from the table of independent events, each session in the sample of sessions including a time-ordered series of independent events occurring within a period of time and mapped to a unique identifier; and
at the time of query execution on each computing device in the cluster of computing devices, analyze, in real time, the sample of sessions to generate a result.
9. The system of claim 8 , wherein the instructions further cause the one or more processors to:
at the time of query execution, process the table of independent events to aggregate a plurality of independent events into the time-ordered series of independent events and reconstruct the sample of sessions based on the time-ordered series of independent events;
transform the sample of sessions into a data structure; and
render a visualization based on the data structure.
10. The system of claim 9 , wherein to determine the query, the instructions further cause the one or more processors to:
receive a user interaction in association with the visualization; and
determine the query based on the user interaction.
11. The system of claim 8 , wherein the instructions further cause the one or more processors to perform funnel analysis using the result.
12. The system of claim 8 , wherein the criteria includes at least one from a timeframe, a set of users, and a geographical location.
13. The system of claim 8 , wherein the unique identifier includes at least one from a session identifier, client device identifier, and a user identifier.
14. The system of claim 8 , wherein the table of independent events is loaded with data retrieved from one of a streaming data source and a batch data source.