Using an electron process to determine a primary indexer for responding to search queries including generation identifiers
Embodiments are directed towards managing within a cluster environment having a plurality of indexers for data storage using redundancy the data being managed using a generation identifier, such that a primary indexer is designated for a given generation of data. When a master device for the cluster fails, data may continue to be stored using redundancy, and data searches performed may still be performed.
1. A computer-implemented method comprising:
generating, at a searcher device, a first query for a set of data for a plurality of indexers in a cluster, wherein the set of data comprises time-stamps within a particular time frame;
providing, with the first query, a first generation identifier for the set of data, wherein the first generation identifier identifies a first indexer from the plurality of indexers to serve as a primary indexer for responding to queries that pertain to the set of data and that comprise the first generation identifier, wherein one or more indexers in the cluster that are other than the primary indexer are designated as secondary indexers, and wherein the secondary indexers are configured to ignore queries that pertain to the set of data and that comprise the first generation identifier;
broadcasting the first query to each indexer in the cluster; and
receiving a response to the first query from the first indexer,
wherein in response to an event prompting a change in a primary indexer designation for the set of data, a second indexer from the secondary indexers is elected as a new primary indexer by an election process conducted between the secondary indexers.
2. The computer-implemented method of claim 1 , wherein, responsive to the election process, the first generation identifier is incremented to generate a second generation identifier, wherein the second generation identifier identifies the second indexer as the new primary indexer for responding to queries that pertain to the set of data.
3. The computer-implemented method of claim 1 , wherein, responsive to the election process, the first generation identifier is incremented to generate a second generation identifier, wherein the second generation identifier identifies the new primary indexer for responding to queries that pertain to the set of data, and further comprising:
generating, at the searcher device, a second query for the set of data;
providing, with the second query, the second generation identifier;
broadcasting the second query to the plurality of indexers in the cluster; and
receiving a response to the second query from the second indexer.
4. The computer-implemented method of claim 1 , wherein the searcher device is comprised within a client device.
5. The computer-implemented method of claim 1 , wherein the searcher device is accessed by a client device.
6. The computer-implemented method of claim 1 , wherein the searcher device is external to the cluster comprising the plurality of indexers.
7. The computer-implemented method of claim 1 , wherein the cluster comprises a master device operable to manage generation identifiers for the cluster.
8. The computer-implemented method of claim 1 , wherein the cluster comprises a master device, wherein the master device is operable to manage generation identifiers for the cluster, and wherein the master device is configured to update the first generation identifier to a second generation identifier, wherein the second generation identifier identifies the second indexer as the new primary indexer for responding to queries that pertain to the set of data.
9. The computer-implemented method of claim 1 , wherein the event prompting a change in the primary indexer designation is a failure of the first indexer.
10. The computer-implemented method of claim 1 , further comprising:
generating, at the searcher device, a second query for the set of data;
waiting for the election process to resolve to receive a response from the second indexer; and
responsive to the election process resolving, receiving a response to the second query from the second indexer.
11. The computer-implemented method of claim 1 , further comprising:
generating, at the searcher device, a second query for the set of data;
providing, with the second query, a plurality of generation identifiers, wherein each generation identifier identifies a different primary indexer for a respective portion of the set of data;
broadcasting the first query to each indexer in the cluster; and
receiving a response from each primary indexer designated by the plurality of generation identifiers for the respective portion of the set of data.
12. The computer-implemented method of claim 1 , wherein the response to the first query comprises metadata, wherein the metadata comprises diagnostic information.
13. The computer-implemented method of claim 1 , wherein the response to the first query comprises metadata, wherein the metadata comprises an amount of time required to process the first query.
14. One or more non-transitory computer-readable storage media, storing software instructions, which when executed by one or more processors cause performance of:
generating, at a searcher device, a first query for a set of data for a plurality of indexers in a cluster, wherein the set of data comprises time-stamps within a particular time frame;
providing, with the first query, a first generation identifier for the set of data, wherein the first generation identifier identifies a first indexer from the plurality of indexers to serve as a primary indexer for responding to queries that pertain to the set of data and that comprise the first generation identifier, wherein one or more indexers in the cluster that are other than the primary indexer are designated as secondary indexers, and wherein the secondary indexers are configured to ignore queries that pertain to the set of data and that comprise the first generation identifier;
broadcasting the first query to each indexer in the cluster; and
receiving a response to the first query from the first indexer,
wherein in response to an event prompting a change in a primary indexer designation for the set of data, a second indexer from the secondary indexers is elected as a new primary indexer by an election process conducted between the secondary indexers.
15. The one or more non-transitory computer-readable storage media of claim 14 , wherein, responsive to the election process, the first generation identifier is incremented to generate a second generation identifier, wherein the second generation identifier identifies the second indexer as the new primary indexer for responding to queries that pertain to the set of data.
16. The one or more non-transitory computer-readable storage media of claim 14 , wherein, responsive to the election process, the first generation identifier is incremented to generate a second generation identifier, wherein the second generation identifier identifies the new primary indexer for responding to queries that pertain to the set of data, and wherein the instructions, when executed by the one or more computing devices, further cause performance of:
generating, at the searcher device, a second query for the set of data;
providing, with the second query, the second generation identifier;
broadcasting the second query to the plurality of indexers in the cluster; and
receiving a response to the second query from the second indexer.
17. The one or more non-transitory computer-readable storage media of claim 14 , wherein the searcher device is comprised within a client device.
18. The one or more non-transitory computer-readable storage media of claim 14 , wherein the searcher device is accessed by a client device.
19. The one or more non-transitory computer-readable storage media of claim 14 , wherein the cluster comprises a master device operable to manage generation identifiers for the cluster.
20. The one or more non-transitory computer-readable storage media of claim 14 , wherein the cluster comprises a master device, wherein the master device is operable to manage generation identifiers for the cluster, and wherein the master device is configured to update the first generation identifier to a second generation identifier, wherein the second generation identifier identifies the second indexer as the new primary indexer for responding to queries that pertain to the set of data.
21. The one or more non-transitory computer-readable storage media of claim 14 , wherein the event prompting a change in the primary indexer designation is a failure of the first indexer.
22. The one or more non-transitory computer-readable storage media of claim 14 , wherein the instructions, when executed by the one or more computing devices, further cause performance of:
generating, at the searcher device, a second query for the set of data;
waiting for the election process to resolve to receive a response from the second indexer; and
responsive to the election process resolving, receiving a response to the second query from the second indexer.
23. The one or more non-transitory computer-readable storage media of claim 14 , wherein the instructions, when executed by the one or more computing devices, further cause performance of:
generating, at the searcher device, a second query for the set of data;
providing, with the second query, a plurality of generation identifiers, wherein each generation identifier identifies a different primary indexer for a respective portion of the set of data;
broadcasting the first query to each indexer in the cluster; and
receiving a response from each primary indexer designated by the plurality of generation identifiers for the respective portion of the set of data.
24. An apparatus comprising:
a subsystem, implemented at least partially in hardware, that generates, at a searcher device, a first query for a set of data for a plurality of indexers in a cluster, wherein the set of data comprises time-stamps within a particular time frame;
a subsystem, implemented at least partially in hardware, that provides, with the first query, a first generation identifier for the set of data, wherein the first generation identifier identifies a first indexer from the plurality of indexers to serve as a primary indexer for responding to queries that pertain to the set of data and that comprise the first generation identifier, wherein one or more indexers in the cluster that are other than the primary indexer are designated as secondary indexers, and wherein the secondary indexers are configured to ignore queries that pertain to the set of data and that comprise the first generation identifier;
a subsystem, implemented at least partially in hardware, that broadcasts the first query to each indexer in the cluster; and
a subsystem, implemented at least partially in hardware, that receives a response to the first query from the first indexer,
wherein in response to an event prompting a change in a primary indexer designation for the set of data, a second indexer from the secondary indexers is elected as a new primary indexer by an election process conducted between the secondary indexers.
25. The apparatus of claim 24 , wherein, responsive to the election process, the first generation identifier is incremented to generate a second generation identifier, wherein the second generation identifier identifies the second indexer as the new primary indexer for responding to queries that pertain to the set of data.
26. The apparatus of claim 24 , wherein, responsive to the election process, the first generation identifier is incremented to generate a second generation identifier, wherein the second generation identifier identifies the new primary indexer for responding to queries that pertain to the set of data, and further comprising:
a subsystem, implemented at least partially in hardware, that generates, at the searcher device, a second query for the set of data;
a subsystem, implemented at least partially in hardware, that provides, with the second query, the second generation identifier;
a subsystem, implemented at least partially in hardware, that broadcasts the second query to the plurality of indexers in the cluster; and
a subsystem, implemented at least partially in hardware, that receives a response to the second query from the second indexer.
27. The apparatus of claim 24 , wherein the searcher device is comprised within a client device.
28. The apparatus of claim 24 , wherein the searcher device is accessed by a client device.
29. The apparatus of claim 24 , wherein the searcher device is external to the cluster comprising the plurality of indexers.
30. The apparatus of claim 24 , wherein the cluster comprises a master device operable to manage generation identifiers for the cluster.