Method and apparatus for efficient synchronization of search heads in a cluster using digests
Embodiments of the present disclosure provide techniques for efficiently and accurately performing propagation of search-head specific configuration customizations across multiple individual configuration files of search heads of a cluster for a consistent user experience. The cluster of search heads may be synchronized such that the search heads operate to receive the configuration or knowledge object customizations from one or more clients from a central or lead search head. To reduce the amount of data that is transferred during propagation, the list of configuration or knowledge object customizations maintained in each search head is filtered from the list of the lead search head until a divergence point is determined. Once determined and communicated to the lead search head, the lead search head sends the configuration and knowledge object customization data that is absent from the internal list of the member search head.
1. A method comprising:
receiving, by one or more processing devices corresponding to a first search head of a cluster comprising a plurality of search heads of a data aggregation and analysis system, a journal entry associated with a knowledge object customization from a second search head in the cluster, wherein the first search head is configured in a leader state and the second search head is configured in a follower state, wherein each search head comprises a configuration file comprising information associated with knowledge objects and knowledge object customizations, and wherein each search head further comprises a journal comprising a history of knowledge object customizations;
adding the journal entry associated with the knowledge object customization to a local data store of the first search head;
updating a first configuration file corresponding to the first search head with the journal entry associated with the knowledge object customization;
updating the journal entry in the first configuration file; and
synchronizing the journal entry with other search heads in the plurality of search heads.
2. The method of claim 1 , wherein the knowledge object customization comprises a customization of a configuration that is one of: a search-related, a visualization-related, or a system-activity-related configuration.
3. The method of claim 1 , wherein the knowledge object customization comprises a customization of a configuration that is one of a search-related configuration or a visualization-related configuration, and wherein the customization may be selected from a group including a deletion, a creation, a modification, a change, or an update of a knowledge object.
4. The method of claim 1 , wherein the knowledge object customization comprises a customization of a knowledge object, and wherein the knowledge object is selected from a group including a saved search, an event type, a transaction, a tag, a field extraction, a field transform, a lookup, a workflow action, a search command, a view, and late-binding schema.
5. The method of claim 1 , wherein receiving the journal entry associated with the knowledge object customization from the second search head in the cluster comprises confirming that a change associated with the journal entry is a latest commit in a respective journal associated with the first search head.
6. The method of claim 1 , wherein adding the journal entry associated with the knowledge object customization to a local data store comprises marking the journal entry to indicate that the journal entry is replicated but not applied to the first configuration file.
7. The method of claim 1 , wherein updating the journal entry in the first configuration file comprises marking the journal entry to indicate that the journal entry is replicated and applied to the first configuration file.
8. The method of claim 1 , further comprising:
transmitting a notification to the second search head that the journal entry was applied at the first search head.
9. The method of claim 1 , wherein synchronizing the journal entry with other search heads in the plurality of search heads comprises performing the synchronizing with the other search heads concurrently.
10. The method of claim 1 , wherein synchronizing the journal entry with other search heads in the plurality of search heads comprises performing the synchronizing with one or more of the other search heads at different times.
11. The method of claim 1 , wherein synchronizing the journal entry with other search heads in the plurality of search heads comprises one or more of the other search heads initializing a synchronization request to retrieve the journal entry from the first search head.
12. A system comprising:
a local data store comprising a first configuration file comprising information associated with knowledge objects and knowledge object customizations, and a first journal comprising a history of knowledge object customizations; and
one or more processing devices coupled with the local data store, the processing devices being configured to:
receive, by the one or more processing devices corresponding to a first search head of a cluster comprising a plurality of search heads of a data aggregation and analysis system, a journal entry associated with a knowledge object customization from a second search head in the cluster, wherein the first search head is configured in a leader state and the second search head is configured in a follower state, wherein each search head comprises a configuration file comprising information associated with knowledge objects and knowledge object customizations, and wherein each search head further comprises a journal comprising a history of knowledge object customizations;
add the journal entry associated with the knowledge object customization to the local data store of the first search head;
update the first configuration file with the journal entry associated with the knowledge object customization;
update the journal entry in the first configuration file; and
synchronize the journal entry with other search heads in the plurality of search heads.
13. The system of claim 12 , wherein the knowledge object customization comprises a customization of a configuration that is one of: a search-related, a visualization-related, or a system-activity-related configuration.
14. The system of claim 12 , wherein the knowledge object customization comprises a customization of a configuration that is one of a search-related configuration or a visualization-related configuration, and wherein the customization may be selected from a group including a deletion, a creation, a modification, a change, or an update of a knowledge object.
15. The system of claim 12 , wherein the knowledge object customization comprises a customization of a knowledge object, and wherein the knowledge object is selected from a group including a saved search, an event type, a transaction, a tag, a field extraction, a field transform, a lookup, a workflow action, a search command, a view, and late-binding schema.
16. The system of claim 12 , wherein receiving the journal entry associated with the knowledge object customization from the second search head in the cluster comprises confirming that a change associated with the journal entry is a latest commit in a respective journal associated with the first search head.
17. The system of claim 12 , wherein adding the journal entry associated with the knowledge object customization to a local data store comprises marking the journal entry to indicate that the journal entry is replicated but not applied to the first configuration file.
18. A non-transitory computer readable medium having instructions stored thereon which, when executed by a processing device, causes the processing device to perform configuration propagation, the instructions comprising:
instructions to receive, by one or more processing devices corresponding to a first search head of a cluster comprising a plurality of search heads of a data aggregation and analysis system, a journal entry associated with a knowledge object customization from a second search head in the cluster, wherein the first search head is configured in a leader state and the second search head is configured in a follower state, wherein each search head comprises a configuration file comprising information associated with knowledge objects and knowledge object customizations, and wherein each search head further comprises a journal comprising a history of knowledge object customizations;
instructions to add the journal entry associated with the knowledge object customization to a local data store of the first search head;
instructions to update a first configuration file corresponding to the first search head with the journal entry associated with the knowledge object customization;
instructions to update the journal entry in the first configuration file; and
instructions to synchronize the journal entry with other search heads in the plurality of search heads.
19. The non-transitory computer readable media of claim 18 , wherein the instructions further comprise:
Instructions to transmit a notification to the second search head that the journal entry was applied at the first search head.
20. The non-transitory computer readable media of claim 18 , wherein instructions to synchronize the journal entry with other search heads in the plurality of search heads comprises instructions to perform the synchronizing with the other search heads concurrently.