Sharing configuration information through a shared storage location
Various embodiments describe multi-site cluster-based data intake and query systems, including cloud-based data intake and query systems. Using a hybrid search system that includes cloud-based data intake and query systems working in concert with so-called “on-premises” data intake and query systems can promote the scalability of search functionality. In addition, the hybrid search system can enable data isolation in a manner in which sensitive data is maintained “on premises” and information or data that is not sensitive can be moved to the cloud-based system. Further, the cloud-based system can enable efficient leveraging of data that may already exist in the cloud.
1. A computer-implemented method comprising:
accessing, by a first cluster of a multi-site data intake and query system, a search configuration that defines at least a portion of how the first cluster retrieves or displays search results, the multi-site data intake and query system comprising the first cluster, a second cluster, and a shared storage location outside the first and second clusters, wherein the first cluster is an on-premises cluster and the second cluster is a cloud-based cluster;
synchronizing, by a first synchronization process that is not triggered by search processing, the search configuration from the first cluster to the shared storage location through a first firewall associated with the first cluster; and
synchronizing, by a second synchronization process that is not triggered by search processing, the search configuration from the shared storage location to the second cluster through a second firewall associated with the second cluster, thereby updating the second cluster based on the search configuration.
2. The computer-implemented method of claim 1 , wherein the search configuration comprises a lookup table configured to enrich data retrieved from a search query.
3. The computer-implemented method of claim 1 , wherein the search configuration comprises extraction rules defining fields of a schema for searching data.
4. The computer-implemented method of claim 1 , wherein the first and second synchronization processes cause the search configuration to be communicated from the first cluster to the second cluster through the shared storage location, and wherein the first and the second clusters are configured to disallow incoming connections.
5. The computer-implemented method of claim 1 , wherein the first and second synchronization processes cause the search configuration to be communicated from the first cluster to the second cluster without direct communication between the first and second clusters.
6. The computer-implemented method of claim 1 , the cloud-based cluster residing in a hosted web service.
7. The computer-implemented method of claim 1 , wherein the shared storage location is accessible by a third cluster configured to access the search configuration from the shared storage location.
8. The computer-implemented method of claim 1 , wherein a first search head of the first cluster is configured to cause the synchronizing of the search configuration from the first cluster to the shared storage location.
9. The computer-implemented method of claim 1 , wherein the search configuration is configured as read-only.
10. The computer-implemented method of claim 1 , wherein the search configuration comprises at least one of saved searches, event types, transactions, tags, field extractions, field transforms, lookups, workflow actions, search commands, or views.
11. One or more non-transitory computer-readable media storing instructions thereon, the instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising:
accessing, by a first cluster of a multi-site data intake and query system, a search configuration that defines at least a portion of how the first cluster retrieves or displays search results, the multi-site data intake and query system comprising the first cluster, a second cluster, and a shared storage location outside the first and second clusters, wherein the first cluster is an on-premises cluster and the second cluster is a cloud-based cluster;
synchronizing, by a first synchronization process that is not triggered by search processing, the search configuration from the first cluster to the shared storage location through a first firewall associated with the first cluster; and
synchronizing, by a second synchronization process that is not triggered by search processing, the search configuration from the shared storage location to the second cluster through a second firewall associated with the second cluster, thereby updating the second cluster based on the search configuration.
12. The one or more non-transitory computer-readable media of claim 11 , wherein the search configuration comprises a lookup table configured to enrich data retrieved from a search query.
13. The one or more non-transitory computer-readable media of claim 11 , wherein the search configuration comprises extraction rules defining fields of a schema for searching data.
14. The one or more non-transitory computer-readable media of claim 11 , wherein the first and second synchronization processes are configured to cause the search configuration to be communicated from the first cluster to the second cluster through the shared storage location, and wherein the first and the second clusters are configured to disallow incoming connections.
15. The one or more non-transitory computer-readable media of claim 11 , wherein the first and second synchronization processes are configured to cause the search configuration to be communicated from the first cluster to the second cluster without direct communication between the first and second clusters.
16. The one or more non-transitory computer-readable media of claim 11 , the cloud-based cluster residing in a hosted web service.
17. The one or more non-transitory computer-readable media of claim 11 , wherein the shared storage location is accessible by a third cluster configured to access the search configuration from the shared storage location.
18. The one or more non-transitory computer-readable media of claim 11 , wherein a first search head of the first cluster is configured to cause the synchronizing of the search configuration from the first cluster to the shared storage location.
19. The one or more non-transitory computer-readable media of claim 11 , wherein the search configuration is configured as read-only.
20. The one or more non-transitory computer-readable media of claim 11 , wherein the search configuration comprises at least one of saved searches, event types, transactions, tags, field extractions, field transforms, lookups, workflow actions, search commands, or views.
21. A computer-implemented system comprising:
one or more processors and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
accessing, by a first cluster of a multi-site data intake and query system, a search configuration that defines at least a portion of how the first cluster retrieves or displays search results, the multi-site data intake and query system comprising the first cluster, a second cluster, and a shared storage location outside the first and second clusters, wherein the first cluster is an on-premises cluster and the second cluster is a cloud-based cluster;
synchronizing, by a first synchronization process that is not triggered by search processing, the search configuration from the first cluster to the shared storage location through a first firewall associated with the first cluster; and
synchronizing, by a second synchronization process that is not triggered by search processing, the search configuration from the shared storage location to the second cluster through a second firewall associated with the second cluster, thereby updating the second cluster based on the search configuration.
22. The computer-implemented system of claim 21 , wherein the search configuration comprises a lookup table configured to enrich data retrieved from a search query.
23. The computer-implemented system of claim 21 , wherein the search configuration comprises extraction rules defining fields of a schema for searching data.
24. The computer-implemented system of claim 21 , wherein the first and second synchronization processes are configured to cause the search configuration to be communicated from the first cluster to the second cluster through the shared storage location, and wherein the first and the second clusters are configured to disallow incoming connections.
25. The computer-implemented system of claim 21 , wherein the first and second synchronization processes are configured to cause the search configuration to be communicated from the first cluster to the second cluster without direct communication between the first and second clusters.
26. The computer-implemented system of claim 21 , the cloud-based cluster residing in a hosted web service.
27. The computer-implemented system of claim 21 , wherein the shared storage location is accessible by a third cluster configured to access the search configuration from the shared storage location.
28. The computer-implemented system of claim 21 , wherein a first search head of the first cluster is configured to cause the synchronizing of the search configuration from the first cluster to the shared storage location.
29. The computer-implemented system of claim 21 , wherein the search configuration is configured as read-only.
30. The computer-implemented system of claim 21 , wherein the search configuration comprises at least one of saved searches, event types, transactions, tags, field extractions, field transforms, lookups, workflow actions, search commands, or views.