IP Library Granted Patent US 11,841,827
Granted Patent B2
US 11,841,827 · App. 17/163,039 · Granted Dec 12, 2023

Facilitating generation of data model summaries

Inventors: Alexandros Batsakis (San Francisco, CA); Ankit Jain (Sunnyvale, CA); Manu Jose (Milpitas, CA); Jonah Pan (San Jose, CA); Hailun Yan (Sunnyvale, CA)
Assignee: Splunk Inc.
G06F16/13G06F16/1824
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,841,827
App. No.
17/163,039
Granted
Dec 12, 2023
Kind
B2
Abstract

Embodiments described herein facilitate enhancement of data model acceleration, including generating data model summaries and performing searches in an accelerated manner. In one implementation, a set of events are indexed, each of the events having a corresponding index time representing a time at which the event was indexed in an indexer. Index time parameters including an index earliest time indicating a first index time at which to begin generating a data model summary and an index latest time indicating a second index time at which to complete generating the data model summary are obtained. Thereafter, a data model summary is generated. Such a data model summary summarizes events having corresponding index times between the index earliest time and the index latest time. The data model summary is provided to a remote data store that is separate from the indexer at which at least a portion of the events were indexed.

Claims (54)

1. A computer-implemented method, comprising:

indexing a set of events, each of the events having a corresponding index time representing a time at which the event was indexed in an indexer;

identifying, at a search head, an indication to generate a data model summary for a data model;

obtaining index time parameters including an index earliest time indicating a first index time at which to begin generating the data model summary and an index latest time indicating a second index time at which to complete generating the data model summary, the first index time and the second index time comprising index times corresponding with the events of the set of events;

generating the data model summary summarizing events having corresponding index times between the index earliest time and the index latest time; and

providing the data model summary to a remote data store for performing a subsequent search using the data model summary, wherein the remote data store that is separate from the search head and the indexer at which at least a portion of the events were indexed.

2. The computer-implemented method of claim 1 wherein the index time parameters are obtained from the search head that determines the index time parameters using event time parameters, index markers, and/or a summarization maximum interval.

3. The computer-implemented method of claim 1 , wherein the index earliest time comprises a marker latest time indicating a last index time associated with an event summarized in a previous data model summary and the index latest time comprises the marker latest time plus a summarization maximum interval indicating a maximum amount of time to use in generating the data model summary.

4. The computer-implemented method of claim 1 , wherein the index earliest time comprises an earliest event time to be included in the data model summary for the data model and the index latest time comprises the earliest event time plus a summarization maximum interval indicating a maximum amount of time to use in generating the data model summary.

5. The computer-implemented method of claim 1 further comprising:

identifying a set of buckets having events associated with the index earliest time through the index latest time; and

using the events in the set of buckets to generate the data model summary.

6. The computer-implemented method of claim 1 further comprising:

obtaining event time parameters including an event earliest time indicating a first event time for generating the data model summary and an event latest time indicating a second event time for generating the data model summary; and

using the event time parameters to generate the data model summary.

7. The computer-implemented method of claim 1 further comprising:

obtaining a staging directory path representing a staging directory at which to store the data model summary, the staging directory comprising a temporary directory used to store the data model summary during processing before data is written to a final directory; and

providing the data model summary to the staging directory at the remote data store based on the staging directory path.

8. The computer-implemented method of claim 1 further comprising:

providing the data model summary to a staging directory at the remote data store the staging directory comprising a temporary directory used during processing; and

providing a summary completion indicator to the search head, wherein the search head moves the data model summary from the staging directory to a final directory at the remote data store.

9. The computer-implemented method of claim 1 , wherein the data model summary is generated in an optimized row columnar (ORC) file format.

10. The computer-implemented method of claim 1 further comprising, at the search head:

creating a staging directory to initially host the data model summary; and

obtaining index markers from the remote data store, the index markers indicating events summarized in a previous data model summary, wherein the index markers are used to determine the index time parameters.

11. The computer-implemented method of claim 1 further comprising, at the search head:

creating a staging directory to initially host the data model summary;

obtaining index markers from the remote data store, the index markers indicating events summarized in a previous data model summary, wherein the index markers are used to determine the index time parameters; and

generating a summarization request that includes a staging directory path and the index time parameters.

12. The computer-implemented method of claim 1 further comprising, at the search head:

receiving a summary completion indicator at the search head;

initiating moving the data model summary from a staging directory to a final directory at the remote data store; and

updating a marker file with a marker earliest time and a marker latest time associated with the data model summary moved to the final directory.

13. The computer-implemented method of claim 1 , wherein the remote data store resides in an external computing service on a different local area network than the indexer.

14. The computer-implemented method of claim 1 , wherein each event in the set of events comprises a time-stamped portion of raw machine data, the raw machine data produced by one or more components within an information technology or security environment and reflects activity within the information technology or security environment.

15. A computing device, comprising:

a processor; and

a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations including:

indexing a set of events, each of the events having a corresponding index time representing a time at which the event was indexed in an indexer;

identifying, at a search head, an indication to generate a data model summary for a data model;

obtaining index time parameters including an index earliest time indicating a first index time at which to begin generating the data model summary and an index latest time indicating a second index time at which to complete generating the data model summary, the first index time and the second index time comprising index times corresponding with the events of the set of events;

generating the data model summary summarizing events having corresponding index times between the index earliest time and the index latest time; and

providing the data model summary to a remote data store for performing a subsequent search using the data model summary, wherein the remote data store that is separate from the search head and the indexer at which at least a portion of the events were indexed.

16. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processor to perform operations including:

indexing a set of events, each of the events having a corresponding index time representing a time at which the event was indexed in an indexer;

identifying, at a search head, an indication to generate a data model summary for a data model;

obtaining index time parameters including an index earliest time indicating a first index time at which to begin generating the data model summary and an index latest time indicating a second index time at which to complete generating the data model summary, the first index time and the second index time comprising index times corresponding with the events of the set of events;

generating the data model summary summarizing events having corresponding index times between the index earliest time and the index latest time; and

providing the data model summary to a remote data store for performing a subsequent search using the data model summary, wherein the remote data store that is separate from the search head and the indexer at which at least a portion of the events were indexed.

17. The non-transitory computer-readable medium of claim 16 , wherein the index earliest time comprises a marker latest time indicating a last index time associated with an event summarized in a previous data model summary and the index latest time comprises the marker latest time plus a summarization maximum interval indicating a maximum amount of time to use in generating the data model summary.

18. The non-transitory computer-readable medium of claim 16 , wherein the index earliest time comprises an earliest event time to be included in the data model summary for the data model and the index latest time comprises the earliest event time plus a summarization maximum interval indicating a maximum amount of time to use in generating the data model summary.

19. The non-transitory computer-readable medium of claim 16 further comprising:

identifying a set of buckets having events associated with the index earliest time through the index latest time; and

using the events in the set of buckets to generate the data model summary.

Assignments (4)
CHANGE OF NAME Recorded Jul 22, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 072170/0599 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2025
From: SPLUNK LLC
To: CISCO TECHNOLOGY, INC.
Reel/Frame 072173/0058 →
CHANGE OF NAME Recorded Jan 6, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 069825/0782 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2021
From: BATSAKIS, ALEXANDROS; JAIN, ANKIT; JOSE, MANU; PAN, JONAH; YAN, HAILUN
To: SPLUNK INC.
Reel/Frame 055101/0808 →
Continuity (1)
Related Publication 20220245091A1 · Aug 4, 2022