Systems and methods for load balancing in a system providing dynamic indexer discovery

1. A method for dynamic load balancing among indexers comprising:

receiving, from an index manager, a status indication associated with a plurality of indexers, wherein each of the plurality of indexers is operable to index events of raw machine-generated data received from a plurality of data collectors;

determining a respective weight associated with each of the plurality of indexers;

selecting an indexer from the plurality of indexers;

allocating data to the indexer in accordance with a respective weight assigned to the indexer; and

transmitting the allocated data to the indexer.

2. The method of claim 1 , wherein the events of raw machine-generated data are each associated with a respective time stamp.

3. The method of claim 1 , wherein the receiving the status indication is responsive to a request transmitted to the index manager.

4. The method of claim 1 , wherein the status indication comprises a link to the status indication maintained at a remote site.

5. The method of claim 1 , wherein the status indication comprises a read-only instantiation of the status indication.

6. The method of claim 1 , wherein the determining comprises:

determining a total value associated with the plurality of indexers; and

determining the respective weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers.

7. The method of claim 1 , wherein the determining comprises:

determining a total value associated with the plurality of indexers; and

determining the respective weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers, wherein the total value comprises a total storage capacity of the plurality of indexers and wherein the performance metric comprises a storage capacity of the indexer.

8. The method of claim 1 , wherein the determining comprises:

determining a total value associated with the plurality of indexers; and

determining the respective weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers, wherein the total value comprises a total storage capacity of the plurality of indexers and wherein the performance metric comprises a storage capacity of the indexer, and wherein the weight associated with the indexer is determined by dividing the storage capacity of the indexer by the total storage capacity of the plurality of indexers.

9. The method of claim 1 , wherein the selecting comprises randomly selecting the indexer from the plurality of indexers.

10. The method of claim 1 , wherein a likelihood of selecting the indexer is based on the weight assigned to the indexer.

11. The method of claim 1 , further comprising:

randomly selecting an indexer from the plurality of indexers at periodic intervals.

12. The method of claim 1 , further comprising:

receiving, from the index manager, the status indication at periodic time intervals, wherein the periodic time intervals are configurable.

13. The method of claim 1 , wherein the receiving the status indication is responsive to a request transmitted to the index manager, and wherein the request comprises a data collector identifier and a site identifier.

14. The method of claim 1 , further comprising:

serializing the allocated data; and

transmitting the serialized allocated data via JavaScript Object Notation (JSON).

15. The method of claim 1 , wherein the transmitting comprises:

establishing a connection with the indexer by transmitting an initial message containing a signature to the indexer; and

responsive to a validation of the signature by the indexer, receiving data back from the indexer.

16. A non-transitory computer-readable medium storing computer-executable instructions which, when executed by a processor, cause the processor to perform operations for dynamic load balancing among indexers comprising:

receiving, from an index manager, a status indication associated with a plurality of indexers, wherein each of the plurality of indexers is operable to index events of raw machine-generated data received from a plurality of data collectors;

determining a respective weight associated with each of the plurality of indexers;

selecting an indexer from the plurality of indexers;

allocating data to the indexer in accordance with a respective weight assigned to the indexer; and

transmitting the allocated data to the indexer.

17. The non-transitory computer-readable medium of claim 16 , wherein the events of raw machine-generated data are each associated with a respective time stamp.

18. The non-transitory computer-readable medium of claim 16 , wherein the status indication comprises a link to the status indication maintained at a remote site.

19. The non-transitory computer-readable medium of claim 16 , wherein the status indication comprises a read-only instantiation of the status indication.

20. The non-transitory computer-readable medium of claim 16 , wherein the determining comprises:

determining a total value associated with the plurality of indexers; and

determining the respective weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers.

21. The non-transitory computer-readable medium of claim 16 , wherein the determining comprises:

determining a total value associated with the plurality of indexers; and

determining the respective weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers, wherein the total value comprises a total storage capacity of the plurality of indexers and wherein the performance metric comprises a storage capacity of the indexer.

22. The non-transitory computer-readable medium of claim 16 , wherein the determining comprises:

determining a total value associated with the plurality of indexers; and

determining the respective weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers, wherein the total value comprises a total storage capacity of the plurality of indexers and wherein the performance metric comprises a storage capacity of the indexer, and wherein the weight associated with the indexer is determined by dividing the storage capacity of the indexer by the total storage capacity of the plurality of indexers.

23. The non-transitory computer-readable medium of claim 16 , wherein the selecting comprises randomly selecting the indexer from the plurality of indexers.

24. The non-transitory computer-readable medium of claim 16 , wherein a likelihood of selecting the indexer is based on the weight assigned to the indexer.

25. The non-transitory computer-readable medium of claim 16 , wherein the receiving the status indication is responsive to a request transmitted to the index manager, and wherein the request comprises a data collector identifier and a site identifier.

26. A system for dynamic load balancing among indexers comprising:

one or more processors; and

one or more memories comprising program instructions stored thereon that are executable by the one or more processors to cause:

receiving at a data collector, from an index manager, a status indication associated with a plurality of indexers, wherein each of the plurality of indexers is operable to index events of raw machine-generated data received from a plurality of data collectors;

determining, by the data collector, a respective weight associated with each of the plurality of indexers;

selecting an indexer from the plurality of indexers; and

transmitting, from the data collector, data to the indexer in accordance with a respective weight assigned to the indexer.

27. The system of claim 26 , wherein the events of raw machine-generated data are each associated with a respective time stamp.

28. The system of claim 26 , wherein the receiving the status indication is responsive to a request transmitted to the index manager from the data collector.

29. The system of claim 26 , wherein the determining comprises:

determining a total value associated with the plurality of indexers; and

determining the respective weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers.

30. The system of claim 26 , wherein the determining comprises:

determining a total value associated with the plurality of indexers; and

determining the respective weight associated with the indexer by using a value of a performance metric of the indexer and the total value associated with the plurality of indexers, wherein the total value comprises a total storage capacity of the plurality of indexers and wherein the performance metric comprises a storage capacity of the indexer.