Systems and methods for providing dynamic indexer discovery
The present disclosure is directed to providing dynamic indexer discovery. An index manager, which may also be known as a cluster master, is configured to track the statuses and capabilities of indexers and provide the statuses and capabilities obtained from the indexers to data collectors, such as forwarders. The data collectors may use the statuses and capabilities associated with the indexers to load balance transmission of data to the indexers. Dynamic indexer discovery may eliminate the need to manually reconfigure data collectors when the status of an indexer changes because the information may be obtained from the index manager without the need to reinitialize the data collectors.
1. A method comprising:
maintaining, by an index manager, a status indication associated with a plurality of indexers, wherein each of the plurality of indexers indexes events of raw machine-generated data received from a plurality of data collectors;
receiving, by the index manager from a data collector of the plurality of data collectors, a request for the status indication associated with the plurality of indexers, wherein the index manager is in communication with one or more of data collectors, and wherein information in the status indication informs the data collector how to determine an allocation of the raw-machine generated data to transmit to the plurality of indexers, and wherein the data collector is operable to determine a manner in which to allocate the raw-machine generated data to the plurality of indexers based on the status indication; and
transmitting, by the index manager, the status indication to the data collector responsive to the request.
2. The method of claim 1 , further comprising:
receiving, by the index manager from a first indexer of the plurality of indexers, a first message indicating a first status of the first indexer and a first performance metric associated with the first indexer;
receiving, by the index manager from a second indexer of the plurality of indexers, a second message indicating a second status of the second indexer and a second performance metric associated with the second indexer;
generating, by the index manager, the status indication based on the first message and the second message; and
transmitting, by the index manager to the data collector in response to the request, the status indication, and wherein information in the status indication informs the data collector how to determine an allocation of the raw-machine generated data to transmit to the first indexer or the second indexer.
3. The method of claim 1 , further comprising:
determining, by the index manager, that a message has not been received from an indexer for a period of time;
determining, by the index manager, that the period of time is greater than a configured threshold, wherein the threshold indicates a maximum period of time before an identified indexer is removed from the status indication; and
removing, by the index manager, the indexer from the status indication.
4. The method of claim 1 , further comprising:
transmitting, by the index manager, the status indication to the plurality of data collectors at periodic time intervals, wherein the periodic time intervals are configurable.
5. The method of claim 1 , further comprising:
transmitting, by the index manager, a notification to the first indexer to transmit status indication messages to the index manager at a pre-determined time.
6. The method of claim 1 , further comprising:
transmitting, by the index manager, a first notification to the data collector specifying a first time interval for the data collector to request the status indication; and
transmitting, by the index manager, a second notification to a second data collector specifying a second time interval for the second data collector to request the status indication, wherein the first time interval differs from the second time interval.
7. The method of claim 1 , wherein the status indication comprises a first performance metric of a first indexer and a second performance metric of a second indexer, and wherein the data collector is configured to use the first performance metric and the second performance metric from the status indication to inform the data collector how to determine an allocation of the raw-machine generated data to transmit to the first indexer and a second portion of the raw machine-generated data to the second indexer.
8. The method of claim 1 , wherein the data collector is configured to receive the status indication from the index manager and does not require reinitializing the data collector to use the status indication.
9. The method of claim 1 , wherein each of the plurality of indexers are configured to transmit a respective message comprising a respective status and a respective performance metric at periodic time intervals, wherein the periodic time intervals are configurable.
10. The method of claim 1 , wherein each of the plurality of indexers are configured to transmit a respective message comprising a respective status and a respective performance metric at a respective pre-determined time.
11. The method of claim 1 , wherein the data collector is configured to identify a first performance metric from the status indication and based on the first performance metric, determine a first portion of the raw-machine generated data to transmit to the first indexer.
12. The method of claim 1 , wherein the status indication comprises a first performance metric and a second performance metric associated with an indexer.
13. The method of claim 1 , wherein the events of raw machine-generated data are obtained from at least one of activity logs, error logs, configuration files, network packets, application data, virtual machine data, or database records.
14. The method of claim 1 , wherein the events of raw machine-generated data are each associated with a respective time stamp.
15. The method of claim 1 , wherein the events of raw machine-generated data are field values extracted using one or more extraction rules of a late-binding schema performed at a search time.
16. The method of claim 1 , wherein the events of raw machine-generated data comprise aggregated heterogeneous machine-generated data generated by at least one of a server, a database, an application, or a network.
17. The method of claim 1 , wherein the events are extracted from raw machine-generated data based at least in part on a real-time search query.
18. The method of claim 1 , further comprising:
transmitting, by the index manager, the status indication to the plurality of indexers at periodic time intervals, wherein the periodic time intervals are configurable.
19. A non-transitory computer-readable medium storing computer-executable instructions which, when executed by a processor, cause the processor to perform operations comprising:
transmitting, to an index manager, a request for a status indication associated with a plurality of indexers, wherein each of the plurality of indexers indexes events of raw machine- generated data received from a plurality of data collectors;
receiving, from the index manager in response to the request, the status indication associated with the plurality of indexers;
selecting an indexer from the plurality of indexers associated with each of the plurality of indexers; and
transmitting data to the indexer, and using information in the status indication to determine an allocation of the raw-machine generated data to transmit to the indexer.
20. The non-transitory computer-readable medium of claim 19 , wherein the events of raw machine-generated data are each associated with a respective time stamp.
21. The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise:
determining a total value associated with the plurality of indexers; and
determining a weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers.
22. The non-transitory computer-readable medium of claim 19 , wherein the request comprises a data collector identifier and a site identifier.
23. The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise:
serializing the data; and
transmitting serialized data via JavaScript Objection Notation (JSON).
24. The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise:
adjusting, based at least in part on the status indication, an amount of data to transmit to the indexer.
25. A non-transitory computer-readable medium storing computer-executable instructions which, when executed by a processor, cause the processor to perform operations comprising:
maintaining a status indication associated with a plurality of indexers, wherein each of the plurality of indexers indexes events of raw machine-generated data received from a plurality of data collectors;
receiving, from a data collector of the plurality of data collectors, a request for the status indication; and
transmitting the status indication to the data collector responsive to the request, and wherein information in the status indication informs the data collector how to determine an allocation of the raw-machine generated data to transmit to the plurality of indexers, and further wherein the data collector is operable to determine a manner in which to allocate the raw-machine generated data to the plurality of indexers based on the status indication.
26. The non-transitory computer-readable medium of claim 25 , wherein each of the plurality of indexers are configured to transmit a respective message comprising a respective status and a respective performance metric at periodic time intervals, wherein the periodic time intervals are configurable.
27. The non-transitory computer-readable medium of claim 25 , wherein each of the plurality of indexers are configured to transmit a respective message comprising a respective status and a respective performance metric at a respective pre-determined time.
28. The non-transitory computer-readable medium of claim 25 , wherein the data collector is configured to identify the first performance metric from the status indication and based on the first performance metric, determine to transmit a first portion of the raw machine-generated data to the first indexer.
29. The non-transitory computer-readable medium of claim 25 , wherein the status indication comprises the first performance metric and the second performance metric.
30. A system comprising:
one or more processors; and
one or more memories comprising program instructions stored thereon that are executable by the one or more processors to cause:
maintaining, by an index manager, a status indication associated with a plurality of indexers, wherein each of the plurality of indexers indexes events of raw machine-generated data received from a plurality of data collectors;
receiving, by the index manager from a data collector of the plurality of data collectors, a request for the status indication associated with the plurality of indexes, wherein the index manager is in communication with one or more of data collectors; and
transmitting, by the index manager, the status indication to the data collector responsive to the request, and wherein information in the status indication informs the data collector how to determine an allocation of the raw-machine generated data to transmit to the plurality of indexers, and further wherein the data collector is operable to determine a manner in which to allocate the raw-machine generated data to the plurality of indexers based on the status indication.