IP Library › Granted Patent US 8,682,886
Granted Patent B2
US 8,682,886 · App. 13/664,239 · Granted Mar 25, 2014

Report acceleration using intermediate summaries of events

Inventors: Stephen Phillip Sorkin (San Francisco, CA); Steve Yu Zhang (San Francisco, CA); Ledion Bitincka (Pasadena, CA)
Assignee: Splunk Inc.
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,682,886
App. No.
13/664,239
Filed
Oct 30, 2012
Granted
Mar 25, 2014
Kind
B2
Art Unit
2155
USPC
707/715
Abstract

A method and system for managing searches of a data set that is partitioned based on a plurality of events. A structure of a search query may be analyzed to determine if logical computational actions performed on the data set is reducible. Data in each partition is analyzed to determine if at least a portion of the data in the partition is reducible. In response to a subsequent or reoccurring search request, intermediate summaries of reducible data and reducible search computations may be aggregated for each partition. Next, a search result may be generated based on at least one of the aggregated intermediate summaries, the aggregated reducible search computations, and a query of adhoc non-reducible data arranged in at least one of the plurality of partitions for the data set.

Claims (102)

1. A computer-implemented method, comprising:

receiving raw data on a computing device;

indexing the raw data, wherein indexing includes dividing the raw data into a set of events;

time stamping each event in the set of events;

storing the set of time stamped events in an indexed data store as event data;

receiving a query to produce a report to be automatically generated and updated with new event data corresponding to time stamped events received during periodic intervals between reports, wherein the query comprises a particular form and structure to produce the report;

determining automatically whether the event data responsive to the query is reducible, wherein reducible event data is used to update the report at the periodic intervals, and wherein the event data responsive to the query is reducible if the event data responsive to the query can be summarized to reduce later processing to generate the report;

in response to determining that the event data responsive to the query is reducible, automatically scheduling generation of one or more intermediate summaries of the event data responsive to the query for distinct time periods in a time range specified by the query for the report, wherein at least one of the distinct time periods occurs during one of the periodic intervals between the reports; and

automatically generating the report at the periodic intervals using at least the one or more intermediate summaries that were generated by analyzing the query to determine that the event data responsive to the query is reducible for generation of the report.

2. The computer-implemented method of claim 1 , wherein automatically generating the report using at least the one or more intermediate summaries includes generating the report from data in the one or more intermediate summaries.

3. The computer-implemented method of claim 1 , wherein automatically generating the report using at least the one or more intermediate summaries further comprises:

determining that additional event data exists that is relevant to the report generated for the query but that was not summarized in the one or more intermediate summaries; and

generating the report using the one or more intermediate summaries and the additional event data.

4. The computer-implemented method of claim 1 , wherein automatically generating the report using at least the one or more intermediate summaries further comprises:

determining that additional event data exists that is relevant to the report generated for the query but that was not summarized in the one or more intermediate summaries, wherein the additional data does not correspond to the distinct time periods; and

generating the report using the one or more intermediate summaries and the additional event data.

5. The computer-implemented method of claim 1 , further comprising:

receiving a second query to produce a second report to be automatically generated, wherein the second query comprises a second particular form and structure to produce the second report;

determining whether the one or more generated intermediate summaries the event data responsive to the query can be reused to generate the second report;

in response to determining that the one or more generated intermediate summaries of the event data responsive to the query can be reused to generate the second report, aggregating the one or more generated intermediate summaries of the event data responsive to the query;

automatically scheduling generation of one or more second intermediate summaries of the event data responsive to the query for a time range specified by the second query for the second report, wherein the one or more second intermediate summaries is generated for the event data that is not used to generate the intermediate summaries; and

automatically generating the second report using the aggregated intermediate summaries or the one or more second intermediate summaries.

6. The computer-implemented method of claim 1 , further comprising:

determining a value for one or more search computation actions to be performed on the event data responsive to the query for generation of the report for the query; and

normalizing the value for the one or more search computation actions before determining automatically whether the event data responsive to the query is reducible.

7. A computer-implemented system, comprising:

one or more processors; and

one or more non-transitory computer-readable storage mediums containing instructions configured to cause the one or more processors to perform operations including:

receiving raw data on a computing device;

indexing the raw data, wherein indexing includes dividing the raw data into a set of events;

time stamping each event in the set of events;

storing the set of time stamped events in an indexed data store as event data;

receiving a query to produce a report to be automatically generated and updated with new event data corresponding to time stamped events received during periodic intervals between reports, wherein the query comprises a particular form and structure to produce the report;

determining automatically whether the event data responsive to the query is reducible, wherein reducible event data is used to update the report at the periodic intervals, and wherein the event data responsive to the query is reducible if the event data responsive to the query can be summarized to reduce later processing to generate the report;

in response to determining that the event data responsive to the query is reducible, automatically scheduling generation of one or more intermediate summaries of the event data responsive to the query for distinct time periods in a time range specified by the query for the report, wherein at least one of the distinct time periods occurs during one of the periodic intervals between the reports; and

automatically generating the report at the periodic intervals using at least the one or more intermediate summaries that were generated by analyzing the query to determine that the event data responsive to the query is reducible for generation of the report.

8. The computer-implemented system of claim 7 , wherein automatically generating the report using at least the one or more intermediate summaries includes generating the report from data in the one or more intermediate summaries.

9. The computer-implemented system of claim 7 , wherein automatically generating the report using at least the one or more intermediate summaries further comprises:

determining that additional event data exists that is relevant to the report generated for the query but that was not summarized in the one or more intermediate summaries; and

generating the report using the one or more intermediate summaries and the additional event data.

10. The computer-implemented system of claim 7 , wherein automatically generating the report using at least the one or more intermediate summaries further comprises:

determining that additional event data exists that is relevant to the report generated for the query but that was not summarized in the one or more intermediate summaries, wherein the additional data does not correspond to the distinct time periods; and

generating the report using the one or more intermediate summaries and the additional event data.

11. The computer-implemented system of claim 7 , wherein the one or more non-transitory computer-readable storage mediums contain further instructions configured to cause the one or more processors to perform operations including:

receiving a second query to produce a second report to be automatically generated, wherein the second query comprises a second particular form and structure to produce the second report;

determining whether the one or more generated intermediate summaries the event data responsive to the query can be reused to generate the second report;

in response to determining that the one or more generated intermediate summaries of the event data responsive to the query can be reused to generate the second report, aggregating the one or more generated intermediate summaries of the event data responsive to the query;

automatically scheduling generation of one or more second intermediate summaries of the event data responsive to the query for a time range specified by the second query for the second report, wherein the one or more second intermediate summaries is generated for the event data that is not used to generate the intermediate summaries; and

automatically generating the second report using the aggregated intermediate summaries or the one or more second intermediate summaries.

12. The computer-implemented system of claim 7 , wherein the one or more non-transitory computer-readable storage mediums contain further instructions configured to cause the one or more processors to perform operations including:

determining a value for one or more search computation actions to be performed on the event data responsive to the query for generation of the report for the query; and

normalizing the value for the one or more search computation actions before determining automatically whether the event data responsive to the query is reducible.

13. A computer-program product, tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to:

receive raw data on a computing device;

index the raw data, wherein indexing includes dividing the raw data into a set of events;

time stamp each event in the set of events;

store the set of time stamped events in an indexed data store as event data;

receive a query to produce a report to be automatically generated and updated with new event data corresponding to time stamped events received during periodic intervals between reports, wherein the query comprises a particular form and structure to produce the report;

determine automatically whether the event data responsive to the query is reducible, wherein reducible event data is used to update the report at the periodic intervals, and wherein the event data responsive to the query is reducible if the event data responsive to the query can be summarized to reduce later processing to generate the report;

in response to determining that the event data responsive to the query is reducible, automatically schedule generation of one or more intermediate summaries of the event data responsive to the query for distinct time periods in a time range specified by the query for the report, wherein at least one of the distinct time periods occurs during one of the periodic intervals between the reports; and

automatically generate the report at the periodic intervals using at least the one or more intermediate summaries that were generated by analyzing the query to determine that the event data responsive to the query is reducible for generation of the report.

14. The computer-program product of claim 13 , wherein automatically generating the report using at least the one or more intermediate summaries includes generating the report from data in the one or more intermediate summaries.

15. The computer-program product of claim 13 , wherein automatically generating the report using at least the one or more intermediate summaries further comprises:

determining that additional event data exists that is relevant to the report generated for the query but that was not summarized in the one or more intermediate summaries; and

generating the report using the one or more intermediate summaries and the additional event data.

16. The computer-program product of claim 13 , wherein automatically generating the report using at least the one or more intermediate summaries further comprises:

determining that additional event data exists that is relevant to the report generated for the query but that was not summarized in the one or more intermediate summaries, wherein the additional data does not correspond to the distinct time periods; and

generating the report using the one or more intermediate summaries and the additional event data.

17. The computer-program product of claim 13 , further including instructions configured to cause the data processing apparatus to:

receive a second query to produce a second report to be automatically generated, wherein the second query comprises a second particular form and structure to produce the second report;

determine whether the one or more generated intermediate summaries of the event data responsive to the query can be reused to generate the second report;

in response to determining that the one or more generated intermediate summaries of the event data responsive to the query can be reused to generate the second report, aggregate the one or more generated intermediate summaries of the event data responsive to the query;

automatically schedule generation of one or more second intermediate summaries of the event data responsive to the query for a time range specified by the second query for the second report, wherein the one or more second intermediate summaries is generated for the event data that is not used to generate the intermediate summaries; and

automatically generate the second report using the aggregated intermediate summaries or the one or more second intermediate summaries.

18. The computer-program product of claim 13 , further including instructions configured to cause the data processing apparatus to:

determine a value for one or more search computation actions to be performed on the event data responsive to the query for generation of the report for the query; and

normalize the value for the one or more search computation actions before determining automatically whether the event data responsive to the query is reducible.

19. A computer-implemented method, comprising:

receiving raw data on a computing device;

indexing the raw data, wherein indexing includes dividing the raw data into a set of events;

time stamping each event in the set of events;

storing the set of time stamped events in an indexed data store as event data;

receiving a query to produce a report to be automatically generated and updated with new event data corresponding to time stamped events received during periodic intervals between reports, wherein the query comprises a particular form and structure to produce the report;

determining automatically whether the event data responsive to the query is reducible, wherein reducible event data is used to update the report at the periodic intervals, and wherein the event data responsive to the query is reducible if the event data responsive to the query can be summarized to reduce later processing to generate the report;

in response to determining that the event data responsive to the query is reducible, automatically scheduling generation of one or more intermediate summaries of the event data responsive to the query for distinct time periods in a time range specified by the query for the report, wherein at least one of the distinct time periods occurs during one of the periodic intervals between the reports; and

automatically generating the report at the periodic intervals using at least the one or more intermediate summaries that were generated by analyzing the query to determine that the event data responsive to the query is reducible for generation of the report.

20. The computer-implemented method of claim 19 , wherein automatically generating the report using at least the one or more intermediate summaries includes generating the report from data in the one or more intermediate summaries.

21. The computer-implemented method of claim 19 , wherein automatically generating the report using at least the one or more intermediate summaries further comprises:

determining that additional event data exists that is relevant to the report generated for the query but that was not summarized in the one or more intermediate summaries; and

generating the report using the one or more intermediate summaries and the additional event data.

22. The computer-implemented method of claim 19 , wherein automatically generating the report using at least the one or more intermediate summaries further comprises:

determining that additional event data exists that is relevant to the report generated for the query but that was not summarized in the one or more intermediate summaries, wherein the additional data does not correspond to the distinct time periods; and

generating the report using the one or more intermediate summaries and the additional event data.

23. The computer-implemented method of claim 19 , further comprising:

receiving a second query to produce a second report to be automatically generated, wherein the second query comprises a second particular form and structure to produce the second report;

determining whether the one or more generated intermediate summaries of the event data responsive to the query can be reused to generate the second report;

in response to determining that the one or more generated intermediate summaries of the event data responsive to the query can be reused to generate the second report, aggregating the one or more generated intermediate summaries of the event data responsive to the query;

automatically scheduling generation of one or more second intermediate summaries of the event data responsive to the query for a time range specified by the second query for the second report, wherein the one or more second intermediate summaries is generated for the event data that is not used to generate the intermediate summaries; and

automatically generating the second report using the aggregated intermediate summaries or the one or more second intermediate summaries.

24. The computer-implemented method of claim 19 , further comprising:

determining a value for one or more search computation actions to be performed on the event data responsive to the query for generation of the report for the query; and

normalizing the value for the one or more search computation actions before determining automatically whether the event data responsive to the query is reducible.

Assignments (3)
CHANGE OF NAME Recorded Jul 22, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 072170/0599 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2025
From: SPLUNK LLC
To: CISCO TECHNOLOGY, INC.
Reel/Frame 072173/0058 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 30, 2012
From: SORKIN, STEPHEN PHILLIP; ZHANG, STEVE YU; BITINCKA, LEDION
To: SPLUNK INC.
Reel/Frame 029213/0663 →
Continuity (2)
Provisional Application 61649125 · May 18, 2012
Related Publication 20130311509A1 · Nov 21, 2013