IP Library › Granted Patent US 12711032
Granted Patent B1
US 12711032 · App. 17/901,586 · Granted Aug 18, 2026

Recovering data from a shared storage system

Inventors: Alexandros Batsakis (San Francisco, CA); Mehul Goyal (Pleasanton, CA); Ashish Mathew (San Mateo, CA); Douglas Rapp (San Mateo, CA); Igor Stojanovski (San Francisco, CA); Eric Woo (San Francisco, CA)
Assignee: Cisco Technology, Inc.
G06F11/3034G06F11/3055G06F11/3075G06F11/326G06F16/1734G06F16/1824
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12711032
App. No.
17/901,586
Granted
Aug 18, 2026
Kind
B1
Abstract

Systems and methods are described for improving data availability and/or resiliency of indexers of a data intake and query system. A data intake and query system can index large amounts of data using one or more indexers. An indexer can store a copy of the data that the indexer is assigned to process in the shared storage system, and a cluster master can track the storage of the data and the indexer assigned to process the data. In the event an indexer fails or is otherwise unable to index data that it has been assigned to index, the cluster master can assign one or more second indexers to process the data. The second indexer can download the data from the shared storage system.

Claims (58)

1 . A method, comprising:

determining, using a first computing device, that a second computing device of a distributed data processing system is not available prior to determining that the second computing device processed a first set of data that the second computing device is assigned to process;

assigning, using the first computing device, a third computing device of the distributed data processing system to process the first set of data based at least in part on determining that the second computing device is not available;

assigning the third computing device to process a second set of data; and

providing the third computing device with location information of the first set of data based at least in part on assigning the third computing device to process the first set of data, the location information indicating a location in a shared storage system at which the first set of data is stored by the second computing device, wherein the third computing device is configured to obtain the first set of data from the location in the shared storage system based at least in part on the location information and process the first set of data and the second set of data to provide results.

2 . The method of claim 1 , further comprising receiving, at the first computing device, the location information from the second computing device.

3 . The method of claim 1 , further comprising receiving, at the first computing device, the location information from the second computing device based at least in part on the second computing device storing the first set of data at the location in the shared storage system.

4 . The method of claim 1 , further comprising receiving, at the first computing device, a data identifier from the second computing device, the data identifier identifying the first set of data.

5 . The method of claim 1 , wherein the first set of data comprises one or more slices of data.

6 . The method of claim 1 , wherein the second computing device is configured to store the first set of data in the shared storage system and store first results of processing the first set of data in at least one of a data store associated with the second computing device or the shared storage system.

7 . The method of claim 1 , further comprising receiving, from the third computing device, an indication that the third computing device successfully processed the first set of data.

8 . The method of claim 1 , wherein said assigning the third computing device is further based at least in part on determining that the second computing device did not process the first set of data.

9 . The method of claim 1 , wherein said determining that the second computing device is not available is based at least in part on at least one of:

absence of communications from the second computing device;

a network failure;

an error associated with the second computing device;

a utilization rate of the second computing device;

an amount of processing resources in use by the second computing device; or

an amount of memory used by the second computing device.

10 . The method of claim 1 , wherein said determining that the second computing device is not available is based on a determination that the second computing device is busy or failing.

11 . The method of claim 1 , further comprising deleting the first set of data from the location in the shared storage system.

12 . The method of claim 1 , wherein the first set of data comprises at least one of raw machine data, structured data, unstructured data, performance metrics data, correlation data, data files, directories of files, data sent over a network, event logs, registries, JSON blobs, XML data, data in a data model, report data, tabular data, messages published to streaming data sources, data exposed in an API, data in a relational database, sensor data, image data, or video data.

13 . The method of claim 1 , wherein the location in the shared storage system is a first location, wherein to process the first set of data, the third computing device is further configured to:

process the first set of data to provide first results; and

upload the first results to a second location in the shared storage system.

14 . The method of claim 1 , further comprising receiving, from the third computing device, an indication that the third computing device successfully processed the first set of data, wherein the location in the shared storage system is a first location, wherein to process the first set of data, the third computing device is further configured to:

process the first set of data to provide the results; and

upload the results to a second location in the shared storage system,

wherein the indication that the third computing device successfully processed the set of first data includes an indication of the second location.

15 . The method of claim 1 , further comprising:

maintaining a database that identifies computing devices of the distributed data processing system, wherein the database associates a particular computing device of the distributed data processing system with a particular set of data assigned to be processed by that particular computing device, wherein the database identifies a particular location of each of the particular sets of data; and

based at least in part on said assigning the third computing device to process the first set of data, updating the database to associate the third computing device with the first set of data and disassociate the second computing device with the first set of data.

16 . A first computing device of a data intake and query system, the first computing device comprising:

memory; and

one or more processors coupled to the memory and configured to:

determine that a second computing device of a distributed data processing system is not available prior to determining that the second computing device processed a first set of data that the second computing device is assigned to process;

assign a third computing device of the distributed data processing system to process the first set of data based at least in part on determining that the second computing device is not available;

assign the third computing device to process a second set of data; and

provide the third computing device with location information of the first set of data based at least in part on assigning the third computing device to process the first set of data, the location information indicating a location in a shared storage system at which the first set of data is stored by the second computing device, wherein the third computing device is configured to obtain the first set of data from the location in the shared storage system based at least in part on the location information and process the first set of data and the second set of data to provide results.

17 . The first computing device of claim 16 , wherein to determine that the second computing device is not available, the one or more processors are further configured to determine that the second computing device is not available based at least in part on at least one of:

absence of communications from the second computing device;

a network failure;

an error associated with the second computing device;

a utilization rate of the second computing device;

an amount of processing resources in use by the second computing device; or

an amount of memory used by the second computing device.

18 . Non-transitory computer readable media comprising computer-executable instructions that, when executed by a first computing device of a data intake and query system, cause the first computing device to:

determine that a second computing device of a distributed data processing system is not available prior to determining that the second computing device processed a first set of data that the second computing device is assigned to process;

assign a third computing device of the distributed data processing system to process the first set of data based at least in part on determining that the second computing device is not available;

assign the third computing device to process a second set of data; and

provide the third computing device with location information of the first set of data based at least in part on assigning the third computing device to process the first set of data, the location information indicating a location in a shared storage system at which the first set of data is stored by the second computing device, wherein the third computing device is configured to obtain the first set of data from the location in the shared storage system based at least in part on the location information and process the first set of data and the second set of data to provide results.

19 . The non-transitory computer readable media of claim 18 , wherein to determine that the second computing device is not available, execution of the computer-executable instructions by the first computing device further cause the first computing device to determine that the second computing device is not available based at least in part on at least one of:

absence of communications from the second computing device;

a network failure;

an error associated with the second computing device;

a utilization rate of the second computing device;

an amount of processing resources in use by the second computing device; or

an amount of memory used by the second computing device.