Search result replication management in a search head cluster
View Patent ↗Systems and methods for search result replication in a search head cluster of a data aggregation and analysis system. An example method may include receiving, by a search head leader of a search head cluster including multiple search heads, from a first search head of the plurality of search heads, a search result in response to a search query. The search head leader parses a registry comprising a set of replicas of the search result in the search head cluster to determine a replication count corresponding to a number of replicas of the search result. A determination is made that the replication count is greater than a target replication count. Based on the determination, a selected replica from the set of replicas is identified based at least in part on a recency of use of the selected replica and a deletion of the selected replica is initiated.
1. A method comprising:
receiving, by a search head leader of a search head cluster comprising a plurality of search heads, from a first search head of the plurality of search heads, a search result in response to a search query, wherein the first search head performs a first reduce phase of a first map-reduce search computation of source data to generate the search result in response to the search query, wherein the search head cluster comprises the plurality of search heads that are each configured to execute a reduce phase of a map-reduce search, and wherein the reduce phase comprises combining results of a map phase of the map-reduce search to generate search results, the map phase executed by a plurality of indexers;
parsing, by the search head leader, a registry comprising a set of replicas of the search result in the search head cluster to determine a replication count corresponding to a number of replicas of the search result;
determining that the replication count is greater than a target replication count stored in a data store;
based on determining that the replication count is greater than the target replication count, identifying a selected replica from the set of replicas to remove based at least in part on the selected replica being associated with a lowest counter value stored in the data store as compared to counter values associated with other replicas from the set of replicas, the lowest counter value indicating a number of times the selected replica was accessed based on incrementing a counter each time the selected replica was accessed; and
transmitting an instruction to delete the selected replica;
wherein the method is performed by one or more processing devices.
2. The method of claim 1 , further comprising receiving, by the search head leader, a statistic associated with the lowest counter value of the selected replica.
3. The method of claim 2 , wherein the statistic is received by the search head leader in response to access of the selected replica of the set of replicas at a search head of the plurality of search heads.
4. The method of claim 1 , further comprising maintaining, by the search head leader, the counter associated with the selected replica of the set of replicas.
5. The method of claim 1 , further comprising deleting the selected replica from the registry.
6. The method of claim 1 , further comprising scheduling the deletion of the selected replica.
7. A computer system, comprising:
a memory; and
one or more processing devices, coupled to the memory, to:
receive, by a search head leader of a search head cluster comprising a plurality of search heads, from a first search head of the plurality of search heads, a search result in response to a search query, wherein the first search head performs a first reduce phase of a first map-reduce search computation of source data to generate the search result in response to the search query, wherein the search head cluster comprises the plurality of search heads that are each configured to execute a reduce phase of a map-reduce search, and wherein the reduce phase comprises combining results of a map phase of the map-reduce search to generate search results, the map phase executed by a plurality of indexers;
parse, by the search head leader, a registry comprising a set of replicas of the search result in the search head cluster to determine a replication count corresponding to a number of replicas of the search result;
determine that the replication count is greater than a target replication count stored in a data store;
based on determining that the replication count is greater than the target replication count, identify a selected replica from the set of replicas to remove based at least in part on the selected replica being associated with a lowest counter value stored in the data store as compared to counter values associated with other replicas from the set of replicas, the lowest counter value indicating a number of times the selected replica was accessed based on incrementing a counter each time the selected replica was accessed; and
transmit an instruction to delete the selected replica.
8. The computer system of claim 7 , wherein the processing device is further to receive a statistic associated with the lowest counter value of the selected replica.
9. The computer system of claim 8 , wherein the statistic is received by the search head leader in response to access of the selected replica of the set of replicas at a search head of the plurality of search heads.
10. The computer system of claim 7 , wherein the processing device is further to maintain the counter associated with the selected replica of the set of replicas.
11. The computer system of claim 7 , wherein the processing device to delete the selected replica from the registry.
12. The computer system of claim 7 , wherein the processing device to schedule the deletion of the selected replica.
13. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a computer system, cause the computer system to perform operations comprising:
receiving, by a search head leader of a search head cluster comprising a plurality of search heads, from a first search head of the plurality of search heads, a search result in response to a search query, wherein the first search head performs a first reduce phase of a first map-reduce search computation of source data to generate the search result in response to the search query, wherein the search head cluster comprises the plurality of search heads that are each configured to execute a reduce phase of a map-reduce search, and wherein the reduce phase comprises combining results of a map phase of the map-reduce search to generate search results, the map phase executed by a plurality of indexers;
parsing, by the search head leader, a registry comprising a set of replicas of the search result in the search head cluster to determine a replication count corresponding to a number of replicas of the search result;
determining that the replication count is greater than a target replication count stored in a data store;
based on determining that the replication count is greater than the target replication count, identifying a selected replica from the set of replicas to remove based at least in part on the selected replica being associated with a lowest counter value stored in the data store as compared to counter values associated with other replicas from the set of replicas, the lowest counter value indicating a number of times the selected replica was accessed based on incrementing a counter each time the selected replica was accessed; and
transmitting an instruction to delete the selected replica.
14. The computer-readable non-transitory storage medium of claim 13 , wherein the operations further comprise receiving a statistic associated with the lowest counter value of the selected replica.
15. The computer-readable non-transitory storage medium of claim 14 , wherein the statistic is received by the search head leader in response to access of the selected replica of the set of replicas at a search head of the plurality of search heads.
16. The computer-readable non-transitory storage medium of claim 13 , wherein the operations further comprise maintaining, by the search head leader, the counter associated with the selected replica of the set of replicas.
17. The computer-readable non-transitory storage medium of claim 13 , wherein the operations further comprise deleting the selected replica from the registry.