IP Library › Granted Patent US 11,275,733

Granted Patent B1

US 11,275,733 · App. 16/513,546 · Granted Mar 15, 2022

Mapping search nodes to a search head using a tenant identifier

Inventors: Alexandros Batsakis (San Francisco, CA); Scott Calvert (El Cerrito, CA); Alexander Douglas James (Sammamish, WA); Bei Li (Redwood City, CA); Ashish Mathew (Daly City, CA); James Monschke (San Francisco, CA); Sogol Moshtaghi (San Francisco, CA); Christopher Madden Pride (Oakland, CA); Xiaowei Wang (Santa Clara, CA)

Assignee: Splunk Inc.

G06F16/24539 G06F16/137 G06F11/3433

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,275,733
App. No.: 16/513,546
Filed: Jul 16, 2019
Granted: Mar 15, 2022
Kind: B1
Examiner: MCQUITERY, DIEDRA M
Art Unit: 2166
USPC: 707/717

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Systems and methods are disclosed for mapping search nodes to a search head in a data intake and query system based on a tenant identifier in order to execute a query received by the data intake and query system. The mapping may allow same or similar search nodes to be used to execute queries that are associated with a particular tenant identifier, in order to take advantage of caching and local data stored with those search nodes. In some cases, search nodes can be mapped based on the tenant identifier using a hashing algorithm, such as a consistent hashing algorithm.

Claims (174)

1. A method comprising:

receiving, by a data intake and query system, a query associated with a tenant identifier, the query identifying a set of data to be processed and a manner of processing the set of data;

identifying, by the data intake and query system, a search head to process the query;

dynamically mapping a first set of one or more search nodes from a plurality of search nodes to the search head based on the tenant identifier associated with the query wherein dynamically mapping the first set of one or more search nodes to the search head is based on receiving the query;

processing the query using the identified search head, wherein processing the query includes generating instructions for the first set of one or more search nodes dynamically mapped to the search head; and

executing the query, wherein executing the query includes communicating the generated instructions to the first set of one or more search nodes dynamically mapped to the search head, wherein the first set of one or more search nodes execute the instructions.

2. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

receiving, by the data intake and query system, a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;

identifying, by the data intake and query system, a different search head to process the second query;

dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein the second set of one or more search nodes includes at least one search node from the first set of one or more search nodes, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head.

3. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein the second set of one or more search nodes has the same search nodes as the first set of one or more search nodes, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head.

4. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein dynamically mapping the first set of one or more search nodes to the search head includes hashing the tenant identifier associated with the query using a hashing algorithm, and wherein dynamically mapping the second set of one or more search nodes to the different search head includes hashing the tenant identifier associated with the second query using the hashing algorithm.

5. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein dynamically mapping the first set of one or more search nodes to the search head includes hashing the tenant identifier associated with the query using a consistent hashing algorithm, and wherein dynamically mapping the second set of one or more search nodes to the different search head includes hashing the tenant identifier associated with the second query using the consistent hashing algorithm.

6. The method of claim 1 , further comprising:

wherein the first set of one or more search nodes execute the query using the first set of buckets mapped to the first set of one or more search nodes.

7. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

mapping a first set of buckets to the first set of one or more search nodes based on a respective node identifier associated with each search node in the first set of one or more search nodes, wherein the first set of buckets includes the first set of data to be processed, wherein the first set of one or more search nodes execute the query using the first set of buckets mapped to the first set of one or more search nodes;

identifying, by the data intake and query system, a different search head to process the second query;

dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

processing the query using the different search head;

mapping a second set of buckets to the second set of one or more search nodes based on a respective node identifier associated with each search node in the second set of one or more search nodes, wherein the second set of buckets includes the second set of data to be processed; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein the second set of one or more search nodes execute the query using the second set of buckets mapped to the second set of one or more search nodes.

8. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

processing the query using the different search head;

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein the second set of one or more search nodes execute the query using the second set of buckets mapped to the second set of one or more search nodes,

wherein mapping the first set of buckets to the first set of one or more search nodes includes hashing the respective node identifier associated with each search node in the first set of one or more search nodes using a hashing algorithm, and wherein mapping the second set of buckets to the second set of one or more search nodes includes hashing the respective node identifier associated with each search node in the second set of one or more search nodes using the hashing algorithm.

9. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

processing the query using the different search head;

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein the second set of one or more search nodes execute the query using the second set of buckets mapped to the second set of one or more search nodes,

wherein mapping the first set of buckets to the first set of one or more search nodes includes hashing the respective node identifier associated with each search node in the first set of one or more search nodes using a consistent hashing algorithm, and wherein mapping the second set of buckets to the second set of one or more search nodes includes hashing the respective node identifier associated with each search node in the second set of one or more search nodes using the consistent hashing algorithm.

10. The method of claim 1 , wherein identifying the search head includes identifying an available search head from a plurality of search heads using data stored by a resource catalog, wherein the resource catalog monitors a status of the plurality of search heads.

11. The method of claim 1 , wherein dynamically mapping the first set of one or more search nodes to the search head includes identifying the plurality of search nodes, wherein the plurality of search nodes satisfy a utilization metric using status data stored by a resource catalog, and identifying one or more search nodes of the plurality of search nodes as the first set of one or more search nodes.

12. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein dynamically mapping the second set of one or more search nodes to the different search head includes identifying the plurality of search nodes, wherein the plurality of search nodes satisfy a utilization metric using status data stored by a resource catalog, and identifying one or more search nodes of the plurality of search nodes as the second set of one or more search nodes.

13. The method of claim 1 , further comprising determining a first maximum number of search nodes for the first set of one or more search nodes and a second maximum number of search nodes for a second set of one or more search nodes.

14. The method of claim 1 , further comprising determining a number of search nodes to include in the first set of one or more search nodes.

15. The method of claim 1 , wherein a number of search nodes in the first set of one or more search nodes is determined based on a configuration associated with the tenant identifier.

16. The method of claim 1 , wherein a number of search nodes in the first set of one or more search nodes is specified by the query.

17. The method of claim 1 , wherein a number of search nodes in the first set of one or more search nodes is determined based on a percentage of available capacity using status data associated with search nodes, wherein the status data is tracked by a resource catalog.

18. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

determining a number of search nodes to include in a second set of one or more search nodes;

dynamically mapping the second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein the second set of one or more search nodes includes at least one search node from the first set of one or more search nodes, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head.

19. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein a number of search nodes in the second set of one or more search nodes is determined based on a configuration associated with the tenant identifier.

20. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein a number of search nodes in the second set of one or more search nodes is specified by the second query.

21. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:

identifying, by the data intake and query system, a different search head to process the second query;

processing the query using the different search head; and

executing the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein a number of search nodes in the second set of one or more search nodes is determined based on a percentage of available capacity using status data associated with search nodes, wherein the status data is tracked by a resource catalog.

22. A system, comprising:

one or more data stores including computer-executable instructions; and

one or more processors configured to:

receive a query associated with a tenant identifier, the query identifying a set of data to be processed and a manner of processing the set of data;

identify a search head to process the query;

dynamically map a first set of one or more search nodes from a plurality of search nodes to the search head based on the tenant identifier associated with the query, wherein dynamically mapping the first set of one or more search nodes to the search head is based on receiving the query;

process the query using the identified search head, wherein processing the query includes generating instructions for the first set of one or more search nodes dynamically mapped to the search node; and

execute the query, wherein executing the query includes communicating the generated instructions to the first set of one or more search nodes dynamically mapped to the search head, wherein the first set of one or more search nodes execute the instructions.

23. The system of claim 22 , wherein the query is a first query and the set of data is a first set of data, wherein the one or more processors are further configured to:

receive a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;

identify a different search head to process the second query;

dynamically map a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein the second set of one or more search nodes includes at least one search node from the first set of one or more search nodes, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

process the query using the different search head; and

execute the query using the second set of one or more search nodes dynamically mapped to the different search head.

24. The system of claim 22 , wherein the query is a first query and the set of data is a first set of data, wherein the one or more processors are further configured to:

receive a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;

identify a different search head to process the second query;

process the query using the different search head; and

execute the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein to dynamically map the first set of one or more search nodes to the search head, the one or more processors are configured to:

hash the tenant identifier associated with the query using a hashing algorithm, and

wherein to dynamically map the second set of one or more search nodes to the different search head, the one or more processors are configured to:

hash the tenant identifier associated with the second query using the hashing algorithm.

25. The system of claim 22 , wherein the query is a first query and the set of data is a first set of data, wherein the one or more processors are further configured to:

receive a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;

identify a different search head to process the second query;

process the query using the different search head; and

execute the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein to dynamically map the first set of one or more search nodes to the search head, the one or more processors are configured to:

hash the tenant identifier associated with the query using a consistent hashing algorithm, and

wherein to dynamically map the second set of one or more search nodes to the different search head, the one or more processors are configured to:

hash the tenant identifier associated with the second query using the consistent hashing algorithm.

26. Non-transitory computer-readable storage media comprising instructions executable by a computing system to:

receive a query associated with a tenant identifier, the query identifying a set of data to be processed and a manner of processing the set of data;

identify a search head to process the query;

27. The non-transitory computer-readable storage media of claim 26 , wherein the query is a first query and the set of data is a first set of data, wherein the instructions are further executable by the computing system to:

receive a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;

identify a different search head to process the second query;

process the query using the different search head; and

execute the query using the second set of one or more search nodes dynamically mapped to the different search head.

28. The non-transitory computer-readable storage media of claim 26 , wherein the query is a first query and the set of data is a first set of data, wherein the instructions are further executable by the computing system to:

receive a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;

identify a different search head to process the second query;

process the query using the different search head; and

execute the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein to dynamically map the first set of one or more search nodes to the search head, the instructions are further executable by the computing system to:

hash the tenant identifier associated with the query using a hashing algorithm, and

wherein to dynamically map the second set of one or more search nodes to the different search head,

the instructions are further executable by the computing system to:

hash the tenant identifier associated with the second query using the hashing algorithm.

29. The non-transitory computer-readable storage media of claim 26 ,

wherein the query is a first query and the set of data is a first set of data, wherein the instructions are further executable by the computing system to:

receive a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;

identify a different search head to process the second query;

dynamically map a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;

process the query using the different search head; and

execute the query using the second set of one or more search nodes dynamically mapped to the different search head,

wherein to dynamically map the first set of one or more search nodes to the search head, the instructions are further executable by the computing system to:

hash the tenant identifier associated with the query using a consistent hashing algorithm, and

wherein to dynamically map the second set of one or more search nodes to the different search head, the instructions are further executable by the computing system to:

hash the tenant identifier associated with the second query using the consistent hashing algorithm.

Assignments (3)

CHANGE OF NAME Recorded Jul 22, 2025

From: SPLUNK INC.

To: SPLUNK LLC

Reel/Frame 072170/0599 →

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2025

From: SPLUNK LLC

To: CISCO TECHNOLOGY, INC.

Reel/Frame 072173/0058 →

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2019

From: BATSAKIS, ALEXANDROS; CALVERT, SCOTT; JAMES, ALEXANDER DOUGLAS; LI, BEI; MATHEW, ASHISH; MONSCHKE, JAMES; MOSHTAGHI, SOGOL; PRIDE, CHRISTOPHER MADDEN; WANG, XIAOWEI

To: SPLUNK INC.

Reel/Frame 050199/0750 →

Continuity (3)

Continuation In Part 16264019 · Jan 31, 2019

Continuation In Part 16147129 · Sep 28, 2018

Continuation In Part 15967587 · Apr 30, 2018

Cited By (13)

US 12,271,911 US 12,299,508 US 12,301,657 US 12,321,396 US 12,346,320 US 12,373,414 US 12,405,948 US 12,475,136 US 12,613,864 US 12,639,379 US 12,670,170 US 12,711,032 US 12,717,795