Clustering for high availability and disaster recovery
Embodiments are directed towards managing within a cluster environment having a plurality of indexers for data storage using redundancy the data being managed using a generation identifier, such that a primary indexer is designated for a given generation of data. When a master device for the cluster fails, data may continue to be stored using redundancy, and data searches performed may still be performed.
1. A computer-implemented method, comprising:
receiving, at an indexer, (i) a first query to search a subset of data accessible to the indexer, and (ii) a first identifier indicating that the indexer has primary responsibility for responding to the first query with a result from a search of the subset of data that is based on the first query;
receiving, at the indexer, (i) a second query to search the subset of data and (ii) a second identifier different from the first identifier and indicating that the indexer does not have primary responsibility for responding to the second query with a result from a search of the subset of data that is based on the second query;
determining, at the indexer, based on the first identifier, and independent from having access to the subset of data, that the indexer should respond to the first query with the result from the search of the subset of data that is based on the first query; and
determining, at the indexer, based on the second identifier, and independent from having access to the subset of data, that the indexer should not respond to the second query with the result from the search of the subset of data that is based on the second query.
2. The method of claim 1 , wherein the first query and the second query are identical.
3. The method of claim 1 , wherein the first query and the second query differ.
4. The method of claim 1 , further comprising:
receiving, at the indexer, raw data;
separating the raw data into a plurality of events included in the subset of data;
determining, for each event in the plurality of events, a time stamp; and
storing each event in the plurality of events in a manner making it searchable by the indexer.
5. The method of claim 1 , further comprising:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data; and
based on storing the subset of data, sending an acknowledgement message.
6. The method of claim 1 , further comprising, prior to receiving the first query:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data;
identifying a replication factor that indicates a number of times that the subset of data is to be replicated; and
forwarding the subset of data to a number of other indexers, wherein the number corresponds to the replication factor.
7. The method of claim 1 , further comprising:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data;
identifying, at the indexer, a replication factor that indicates a number of times that the subset of data is to be replicated;
forwarding, by the indexer, the subset of data to a number of other indexers, wherein the number corresponds to the replication factor;
determining that an acknowledgement receipt, responsive to the forwarding the subset of data, has not been received from at least one of the other indexers; and
upon determining that the acknowledgement receipt has not been received, forwarding the subset of data to another indexer not in the other indexers.
8. The method of claim 1 , wherein a copy of the subset of data is accessible by at least one other indexer, and wherein the first identifier indicates that the at least one other indexer does not have primary responsibility for responding to the first query with the result from the search of the subset of data that is based on the first query.
9. The method of claim 1 , wherein the indexer determines that the indexer should respond to the first query with the result from the search of the subset of data that is based on the first query by comparing the first identifier with a notification from a master about when the indexer should respond to a query associated with a given identifier.
10. A system comprising:
one or more data processors; and
a non-transitory computer-readable storage medium containing instructions which when executed on the one or more data processors, cause the one or more processors to perform operations including:
receiving, at an indexer, (i) a first query to search a subset of data accessible to the indexer, and (ii) a first identifier indicating that the indexer has primary responsibility for responding to the first query with a result from a search of the subset of data that is based on the first query;
receiving, at the indexer, (i) a second query to search the subset of data and (ii) a second identifier different from the first identifier and indicating that the indexer does not have primary responsibility for responding to the second query with a result from a search of the subset of data that is based on the second query;
determining, at the indexer, based on the first identifier, and independent from having access to the subset of data, that the indexer should respond to the first query with the result from the search of the subset of data that is based on the first query; and
determining, at the indexer, based on the second identifier, and independent from having access to the subset of data, that the indexer should not respond to the second query with the result from the search of the subset of data that is based on the second query.
11. The system of claim 10 , wherein the first query and the second query are identical.
12. The system of claim 10 , wherein the first query and the second query differ.
13. The system of claim 10 , the operations further comprising:
receiving, at the indexer, raw data;
separating the raw data into a plurality of events included in the subset of data;
determining, for each event in the plurality of events, a time stamp; and
storing each event in the plurality of events in a manner making it searchable by the indexer.
14. The system of claim 10 , the operations further comprising:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data; and
based on storing the subset of data, sending an acknowledgement message.
15. The system of claim 10 , the operations further comprising, prior to receiving the first query:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data;
identifying a replication factor that indicates a number of times that the subset of data is to be replicated; and
forwarding the subset of data to a number of other indexers, wherein the number corresponds to the replication factor.
16. The system of claim 10 , the operations further comprising:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data;
identifying, at the indexer, a replication factor that indicates a number of times that the subset of data is to be replicated;
forwarding, by the indexer, the subset of data to a number of other indexers, wherein the number corresponds to the replication factor;
determining that an acknowledgement receipt, responsive to the forwarding the subset of data, has not been received from at least one of the other indexers; and
upon determining that the acknowledgement receipt has not been received, forwarding the subset of data to another indexer not in the other indexers.
17. The system of claim 10 , wherein a copy of the subset of data is accessible by at least one other indexer, and wherein the first identifier indicates that the at least one other indexer does not have primary responsibility for responding to the first query with the result from the search of the subset of data that is based on the first query.
18. The system of claim 10 , wherein the indexer determines that the indexer should respond to the first query with the result from the search of the subset of data that is based on the first query by comparing the first identifier with a notification from a master about when the indexer should respond to a query associated with a given identifier.
19. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions to cause one or more data processors perform operations including:
receiving, at an indexer, (i) a first query to search a subset of data accessible to the indexer, and (ii) a first identifier indicating that the indexer has primary responsibility for responding to the first query with a result from a search of the subset of data that is based on the first query;
receiving, at the indexer, (i) a second query to search the subset of data and (ii) a second identifier different from the first identifier and indicating that the indexer does not have primary responsibility for responding to the second query with a result from a search of the subset of data that is based on the second query;
determining, at the indexer, based on the first identifier, and independent from having access to the subset of data, that the indexer should respond to the first query with the result from the search of the subset of data that is based on the first query; and
determining, at the indexer, based on the second identifier, and independent from having access to the subset of data, that the indexer should not respond to the second query with the result from the search of the subset of data that is based on the second query.
20. The computer-program product of claim 19 , wherein the first query and the second query are identical.
21. The computer-program product of claim 19 , wherein the first query and the second query differ.
22. The computer-program product of claim 19 , the operations further comprising:
receiving, at the indexer, raw data;
separating the raw data into a plurality of events included in the subset of data;
determining, for each event in the plurality of events, a time stamp; and
storing each event in the plurality of events in a manner making it searchable by the indexer.
23. The computer-program product of claim 19 , the operations further comprising:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data; and
based on storing the subset of data, sending an acknowledgement message.
24. The computer-program product of claim 19 , the operations further comprising, prior to receiving the first query:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data;
identifying a replication factor that indicates a number of times that the subset of data is to be replicated; and
forwarding the subset of data to a number of other indexers, wherein the number corresponds to the replication factor.
25. The computer-program product of claim 19 , the operations further comprising:
receiving, at the indexer, the subset of data;
storing, by the indexer, the subset of data;
identifying, at the indexer, a replication factor that indicates a number of times that the subset of data is to be replicated;
forwarding, by the indexer, the subset of data to a number of other indexers, wherein the number corresponds to the replication factor;
determining that an acknowledgement receipt, responsive to the forwarding the subset of data, has not been received from at least one of the other indexers; and
upon determining that the acknowledgement receipt has not been received, forwarding the subset of data to another indexer not in the other indexers.
26. The computer-program product of claim 19 , wherein a copy of the subset of data is accessible by at least one other indexer, and wherein the first identifier indicates that the at least one other indexer does not have primary responsibility for responding to the first query with the result from the search of the subset of data that is based on the first query.
27. The computer-program product of claim 19 , wherein the indexer determines that the indexer should respond to the first query with the result from the search of the subset of data that is based on the first query by comparing the first identifier with a notification from a master about when the indexer should respond to a query associated with a given identifier.
28. The method of claim 1 , wherein the search of the subset of data that is based on the first query is identical to the search of the subset of data that is based on the second query.
29. The method of claim 1 , wherein the search of the subset of data that is based on the first query differs from the search of the subset of data that is based on the second query.
30. The method of claim 1 , wherein the first query also requires a search of a second subset of data accessible to the indexer, and wherein the first identifier also indicates that the indexer does not have primary responsibility for responding to the first query with a result from a search of the second subset of data that is based on the first query.