Dynamically assigning a search head to process a query
Systems and methods are disclosed for mapping search nodes to a search head in a data intake and query system based on a tenant identifier in order to execute a query received by the data intake and query system. The mapping may allow same or similar search nodes to be used to execute queries that are associated with a particular tenant identifier, in order to take advantage of caching and local data stored with those search nodes. In some cases, search nodes can be mapped based on the tenant identifier using a hashing algorithm, such as a consistent hashing algorithm.
1. A method comprising:
receiving, by a data intake and query system, a first query associated with a first tenant, wherein the first query identifies a set of data to be processed and a manner of processing the set of data, the set of data stored in one or more buckets, each of the one or more buckets storing raw machine data associated with a timestamp; and
dynamically assigning, by the data intake and query system, a search head of a plurality of search heads to process the first query,
wherein the search head is instantiated in an isolated execution environment on a computing device in a hosted computing environment,
wherein the search head concurrently processes the first query and a second query associated with a second tenant, and
wherein the first query is executed by a first group of search nodes instantiated in a first group of isolated execution environments and the second query is executed by a second group of search nodes instantiated in a second group of isolated execution environments.
2. The method of claim 1 , wherein the search head comprises a containerized search head.
3. The method of claim 1 , wherein the first group of search nodes comprises a first group of containerized search nodes and the second group of search nodes comprises a second group of containerized search nodes.
4. The method of claim 1 , wherein the search head is instantiated as a container.
5. The method of claim 1 , wherein the first query is dynamically assigned to be executed by the first group of search nodes and the second query is dynamically assigned to be executed by the second group of search nodes.
6. The method of claim 1 , wherein the first group of search nodes is instantiated in the first group of isolated execution environments on one or more first computing devices and the second group of search nodes is instantiated in the second group of isolated execution environments on one or more second computing devices.
7. The method of claim 1 , wherein the set of data comprises a set of data of the first tenant.
8. The method of claim 1 , wherein the first group of search nodes and the second group of search nodes overlap, such that a common set of search nodes includes one or more search nodes in both the first group of search nodes and the second group of search nodes.
9. The method of claim 1 , further comprising periodically monitoring, by the data intake and query system, one or more search head utilization metrics associated with the plurality of search heads, wherein the one or more search head utilization metrics comprise at least one of:
a processor utilization metric for each search head in the plurality of search heads;
a number of queries being processed by each search head in the plurality of search heads; or
a concurrent number of queries being processed among the plurality of search heads.
10. The method of claim 1 , wherein the hosted computing environment is a containerized environment.
11. The method of claim 1 , further comprising periodically monitoring, by the data intake and query system, one or more search node utilization metrics associated with a plurality of search nodes, wherein the first group of search nodes and the second group of search nodes belong to the plurality of search nodes.
12. The method of claim 1 , further comprising periodically monitoring, by the data intake and query system, one or more search node utilization metrics associated with a plurality of search nodes, wherein the first group of search nodes and the second group of search nodes belong to the plurality of search nodes, wherein the one or more search node utilization metrics comprise at least one of:
a processor utilization metric for each search node in the plurality of search nodes; or
a memory utilization metric for each search node in the plurality of search nodes.
13. The method of claim 1 , further comprising periodically monitoring, by the data intake and query system, one or more search node utilization metrics associated with a plurality of search nodes, wherein the first group of search nodes and the second group of search nodes belong to the plurality of search nodes, wherein one or more of the first group of search nodes or the second group of search nodes are instantiated based on the one or more search node utilization metrics associated with the plurality of search nodes.
14. The method of claim 1 , further comprising:
periodically monitoring, by the data intake and query system, one or more search node utilization metrics associated with a plurality of search nodes, wherein the first group of search nodes and the second group of search nodes belong to the plurality of search nodes; and
instantiating, by the data intake and query system, an additional search node, for the plurality of search nodes, based on the one or more search node utilization metrics associated with the plurality of search nodes.
15. The method of claim 1 , further comprising:
periodically monitoring, by the data intake and query system, one or more search node utilization metrics associated with a plurality of search nodes, wherein the plurality of search nodes comprises the first group of search nodes and the second group of search nodes;
determining, by the data intake and query system, based at least in part on periodically monitoring the one or more search node utilization metrics, a most-recent set of the one or more search node utilization metrics associated with the plurality of search nodes; and
identifying, by the data intake and query system, the first group of search nodes from the plurality of search nodes based on the most-recent set of the one or more search node utilization metrics associated with the plurality of search nodes.
16. The method of claim 1 , further comprising:
periodically monitoring, by the data intake and query system, one or more search node utilization metrics associated with a plurality of search nodes, wherein the plurality of search nodes comprises the first group of search nodes and the second group of search nodes;
determining, by the data intake and query system, based at least in part on periodically monitoring the one or more search node utilization metrics, a most-recent set of the one or more search node utilization metrics associated with the plurality of search nodes;
determining, by the data intake and query system, a search node assignment number; and
identifying, by the data intake and query system, the first group of search nodes from the plurality of search nodes based on the most-recent set of the one or more search node utilization metrics associated with the plurality of search nodes and the search node assignment number.
17. The method of claim 1 , further comprising:
periodically monitoring, by the data intake and query system, one or more search node utilization metrics associated with a plurality of search nodes, wherein the plurality of search nodes comprises the first group of search nodes and the second group of search nodes;
determining, by the data intake and query system, based at least in part on periodically monitoring the one or more search node utilization metrics, a most-recent set of the one or more search node utilization metrics associated with the plurality of search nodes;
determining, by the data intake and query system, a search node assignment number; and
identifying, by the data intake and query system, the first group of search nodes from the plurality of search nodes based on the most-recent set of the one or more search node utilization metrics associated with the plurality of search nodes and the search node assignment number, wherein the search node assignment number specifies one or more of:
a number of search nodes to assign to execute at least a portion of the first query, and wherein the first group of search nodes is identified from the plurality of search nodes such that the first group of search nodes has a number of search nodes equal to the search node assignment number, or
a maximum number of search nodes to assign to execute at least a portion of the first query, and wherein the first group of search nodes is identified from the plurality of search nodes such that the first group of search nodes has a number of search nodes equal to, or less than, the search node assignment number.
18. The method of claim 1 , further comprising:
periodically monitoring, by the data intake and query system, one or more search node utilization metrics associated with a plurality of search nodes, wherein the plurality of search nodes comprises the first group of search nodes and the second group of search nodes;
determining, by the data intake and query system, based at least in part on periodically monitoring the one or more search node utilization metrics, a most-recent set of the one or more search node utilization metrics associated with the plurality of search nodes;
determining, by the data intake and query system, a search node assignment number; and
identifying, by the data intake and query system, the first group of search nodes from the plurality of search nodes based on the most-recent set of the one or more search node utilization metrics associated with the plurality of search nodes and the search node assignment number, wherein the search node assignment number is one or more of:
a static value specified in a configuration,
a value associated with the first tenant,
a value determined based on a total number of search nodes in the plurality of search nodes and the one or more search node utilization metrics associated with the plurality of search nodes, or
a value specified by the first query.
19. A system, comprising:
one or more data stores including computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions, wherein execution of the computer-executable instructions causes the system to:
receive a first query associated with a first tenant, wherein the first query identifies a set of data to be processed and a manner of processing the set of data, the set of data stored in one or more buckets, each of the one or more buckets storing raw machine data associated with a timestamp; and
dynamically assign a search head of a plurality of search heads to process the first query,
wherein the search head is instantiated in an isolated execution environment on a computing device in a hosted computing environment,
wherein the search head concurrently processes the first query and a second query associated with a second tenant, and
wherein the first query is executed by a first group of search nodes instantiated in a first group of isolated execution environments and the second query is executed by a second group of search nodes instantiated in a second group of isolated execution environments.
20. Non-transitory computer-readable storage media comprising instructions executable by a computing system to:
receive a first query associated with a first tenant, wherein the first query identifies a set of data to be processed and a manner of processing the set of data, the set of data stored in one or more buckets, each of the one or more buckets storing raw machine data associated with a timestamp; and
dynamically assign a search head of a plurality of search heads to process the first query,
wherein the search head is instantiated in an isolated execution environment on a computing device in a hosted computing environment,
wherein the search head concurrently processes the first query and a second query associated with a second tenant, and
wherein the first query is executed by a first group of search nodes instantiated in a first group of isolated execution environments and the second query is executed by a second group of search nodes instantiated in a second group of isolated execution environments.