IP Library Granted Patent US 12,645,382
Granted Patent B2
US 12,645,382 · App. 18/618,772 · Granted Jun 2, 2026

Collector module coordination for ingestion of data from a data source

Inventors: Mehrdad Arian (Vancouver, CA); Omprakaash Thoppai (San Jose, CA); Muzhi Ou (Vancouver, CA)
Assignee: Cisco Technology, Inc.
G06F3/064G06F3/0604G06F3/0656G06F3/0673
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,645,382
App. No.
18/618,772
Granted
Jun 2, 2026
Kind
B2
Abstract

A host computing device can receive an instruction to collect data from a data source and can determine a type of the data source. Based on the type of the data source, the host computing device can assign a plurality of isolated execution environments to generate a cluster of collector modules to collect data from the data source and can identify a collector module as a cluster leader. The cluster leader can obtain a data identifier of a plurality of blocks of data to be collected and can communicate the data identifier of the plurality of blocks of data to a communication buffer. The cluster of collector modules can retrieve the plurality of blocks of data from the data source using the data identifier of the plurality of blocks of data.

Claims (70)

1 . A method, comprising:

receiving an instruction to collect data from a first data source;

determining a type of the first data source is a first type;

based on a determination that the type of the first data source is the first type:

assigning a plurality of isolated execution environments to generate a first cluster of collector modules to collect the data from the first data source, wherein each of the plurality of isolated execution environments instantiates at least one collector module of the first cluster of collector modules and configures the at least one collector module to collect the data from the first data source, and

assigning a first collector module of the first cluster of collector modules as a cluster leader, wherein the cluster leader is configured to:

obtain a data identifier of a plurality of blocks of data to be collected by the first cluster of collector modules, and

communicate the data identifier of the plurality of blocks of data to a communication buffer, wherein the first cluster of collector modules is configured to retrieve the data identifier of the plurality of blocks of data from the communication buffer; and

retrieving, using the first cluster of collector modules, the plurality of blocks of data from the first data source.

2 . The method of claim 1 , further comprising:

retrieving configuration data from the communication buffer based on the instruction, wherein assigning the plurality of isolated execution environments is based on the configuration data.

3 . The method of claim 1 , further comprising:

retrieving configuration data from the communication buffer based on the instruction, wherein assigning the plurality of isolated execution environments is based on the configuration data, wherein the configuration data indicates the type of the first data source.

4 . The method of claim 1 , further comprising:

retrieving configuration data from the communication buffer based on the instruction, wherein assigning the plurality of isolated execution environments is based on the configuration data, wherein the configuration data indicates an identifier of a connector corresponding to the first cluster of collector modules.

5 . The method of claim 1 , further comprising:

retrieving configuration data from the communication buffer based on the instruction, wherein assigning the plurality of isolated execution environments is based on the configuration data, wherein the configuration data indicates a number of collector modules to be instantiated for the first cluster of collector modules.

6 . The method of claim 1 , further comprising:

retrieving configuration data from the communication buffer based on the instruction, wherein assigning the plurality of isolated execution environments is based on the configuration data, wherein determining the type of the first data source is the first type is based on the configuration data.

7 . The method of claim 1 , wherein assigning the plurality of isolated execution environments to generate the first cluster of collector modules comprises:

assigning at least one of the plurality of isolated execution environments to generate at least two collector modules of the first cluster of collector modules.

8 . The method of claim 1 , wherein each of the plurality of isolated execution environments executes a respective plurality of collector modules corresponding to a respective plurality of connectors.

9 . The method of claim 1 , wherein the plurality of isolated execution environments is instantiated on and share compute resources of a host computing device.

10 . The method of claim 1 , wherein a first isolated execution environment of the plurality of isolated execution environments generates the at least one collector module of the first cluster of collector modules, configures the at least one collector module of the first cluster of collector modules to collect the data from the first data source, generates a second collector module of a second cluster of collector modules, and configures the second collector module of the second cluster of collector modules to collect second data from a second data source.

11 . The method of claim 1 , wherein a second type of data source is configured to support parallel reads.

12 . The method of claim 1 , wherein determining the type of the first data source is the first type is based on the determination that the first data source is not configured for parallel reads.

13 . The method of claim 1 , further comprising:

retrieving the data identifier of the plurality of blocks of data from the communication buffer to obtain a retrieved data identifier of the plurality of blocks of data, wherein retrieving the plurality of blocks of data from the first data source is based on the retrieved data identifier of the plurality of blocks of data.

14 . The method of claim 1 , wherein retrieving the plurality of blocks of data from the first data source comprises:

retrieving a first data identifier of a block of data from the communication buffer; and

requesting, using the first data identifier of the block of data, the block of data from the first data source.

15 . The method of claim 1 , wherein retrieving the plurality of blocks of data from the first data source comprises:

retrieving a first data identifier of a block of data from the communication buffer; and

requesting, using the first data identifier of the block of data, the block of data from the first data source,

the method further comprising:

processing the block of data.

16 . The method of claim 1 , wherein retrieving the plurality of blocks of data from the first data source comprises:

retrieving a first data identifier of a block of data from the communication buffer; and

requesting, using the first data identifier of the block of data, the block of data from the first data source,

the method further comprising:

processing the block of data; and

providing an update to the communication buffer indicating processing of the block of data.

17 . The method of claim 1 , further comprising:

receiving a second instruction to collect second data from a second data source;

determining a type of the second data source is a second type; and

based on determining the type of the second data source is the second type, retrieving, using a second cluster of collector modules, a second plurality of blocks of second data from the second data source.

18 . The method of claim 1 , further comprising:

receiving a second instruction to collect second data from a second data source;

determining a type of the second data source is a second type; and

based on a determination that the type of the second data source is the second type, retrieving, using a second cluster of collector modules, a second plurality of blocks of second data from the second data source, wherein the second data source provides coordination of the second cluster of collector modules.

19 . A host computing device comprising:

a data store; and

one or more processors configured to:

receive an instruction to collect data from a first data source;

determine a type of the first data source is a first type;

based on a determination that the type of the first data source is the first type:

assign a plurality of isolated execution environments to generate a first cluster of collector modules to collect the data from the first data source, wherein each of the plurality of isolated execution environments instantiates at least one collector module of the first cluster of collector modules and configures the at least one collector module to collect the data from the first data source, and

assign a first collector module of the first cluster of collector modules as a cluster leader, wherein the cluster leader is configured to:

obtain a data identifier of a plurality of blocks of data to be collected by the first cluster of collector modules, and

communicate the data identifier of the plurality of blocks of data to a communication buffer, wherein the first cluster of collector modules is configured to retrieve the data identifier of the plurality of blocks of data from the communication buffer; and

retrieve, using the first cluster of collector modules, the plurality of blocks of data from the first data source.

20 . One or more non-transitory computer-readable media including computer-executable instructions that, when executed by a host computing device, cause the host computing device to:

receive an instruction to collect data from a first data source;

determine a type of the first data source is a first type;

based on a determination that the type of the first data source is the first type:

assign a plurality of isolated execution environments to generate a first cluster of collector modules to collect the data from the first data source, wherein each of the plurality of isolated execution environments instantiates at least one collector module of the first cluster of collector modules and configures the at least one collector module to collect the data from the first data source, and

assign a first collector module of the first cluster of collector modules as a cluster leader, wherein the cluster leader is configured to:

obtain a data identifier of a plurality of blocks of data to be collected by the first cluster of collector modules, and

communicate the data identifier of the plurality of blocks of data to a communication buffer, wherein the first cluster of collector modules is configured to retrieve the data identifier of the plurality of blocks of data from the communication buffer; and

retrieve, using the first cluster of collector modules, the plurality of blocks of data from the first data source.

Assignments (4)
CHANGE OF NAME Recorded Jul 22, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 072170/0599 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2025
From: SPLUNK LLC
To: CISCO TECHNOLOGY, INC.
Reel/Frame 072173/0058 →
CHANGE OF NAME Recorded Jan 6, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 069826/0065 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 4, 2024
From: ARIAN, MEHRDAD; THOPPAI, OMPRAKAASH; OU, MUZHI
To: SPLUNK INC.
Reel/Frame 067006/0640 →
Continuity (1)
Related Publication 20250306787A1 · Oct 2, 2025
References Cited (14)
US 10262271B1 · Doddi · 2019 [cited by examiner]
US 10613897B1 · Wang et al. · 2020 [cited by applicant]
US 10929415B1 · Shcherbakov · 2021 [cited by examiner]
US 11513846B1 · Fletcher · 2022 [cited by examiner]
US 20090106780A1 · Nord et al. · 2009 [cited by applicant]
US 20200364223A1 · Pal · 2020 [cited by examiner]
US 20220014451A1 · Naik · 2022 [cited by examiner]
US 20220103349A1 · Tsirkin et al. · 2022 [cited by applicant]
US 20220247819A1 · Tornow et al. · 2022 [cited by applicant]
US 20220413943A1 · Poornachandran · 2022 [cited by examiner]
US 20230106455A1 · Bursell et al. · 2023 [cited by applicant]
US 20230188516A1 · Danilov et al. · 2023 [cited by applicant]
International Search Report and Written Opinion Mailed Feb. 4, 2025 in International Patent Application No. PCT/US2024/039050. 12 pages. [cited by applicant]
Mickael et al. “Kafka connect: Build and run data pipelines” Published Oct. 3, 2023 and retrieved Dec. 16, 2024 from the Internet at <https://www.redhat.com/rhdc/managed-files/cl-oreilly-kafka-connect-pipelines-analyst-… [cited by applicant]