IP Library › Granted Patent US 10,678,805
Granted Patent B2
US 10,678,805 · App. 15/966,279 · Granted Jun 9, 2020

Schedule modification of data collection requests sent to external data sources

Inventors: Ken Chen (Shanghai, CN); Gang Tao (Shanghai, CN); Lai Qiang Ding (Nanjing, CN); Junqing Hao (Nanjing, CN); Ting Wang (Shanghai, CN); Elias Haddad (San Francisco, CA); Dritan Bitincka (Edgewater, NJ)
Assignee: Splunk Inc.
G06F16/248H04L43/024H04L43/0817
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,678,805
App. No.
15/966,279
Filed
Apr 30, 2018
Granted
Jun 9, 2020
Kind
B2
Art Unit
2444
USPC
709/224
Abstract

Techniques and mechanisms are disclosed that enable a data collection system to adaptively control collection of data from one or more external data sources. At a high level, adaptively controlling collection of data from external data sources may include collecting performance information related to one or more data collection nodes and, in response to analyzing the collected performance information, adapting rates at which the data collection nodes send data collection requests to external data sources. Data collection performance information generally may include, but is not limited to, network traffic data, error messages generated by external data sources and/or data collection nodes, computing device performance information, and any other types of information related to a data collection node's ability to collect data from external data sources.

Claims (53)

1. A method, comprising:

receiving, by a controller node, data collection performance information from a data collection node, the data collection performance information related to data collection requests sent by the data collection node to at least one component of a plurality of components of a hosted computing environment, the plurality of components of the hosted computing environment including distributed computing devices used to generate computing environments that share compute resources of a particular distributed computing device of the distributed computing devices;

generating instructions for the data collection node to modify a schedule according to which the data collection node sends data collection requests to the at least one component of the hosted computing environment, wherein the instructions to modify the schedule are generated based on the received data collection performance information; and

sending the instructions to the data collection node.

2. The method of claim 1 , wherein the schedule includes a data collection request rate limit for the data collection node to send data collection requests to the at least one component of the hosted computing environment.

3. The method of claim 1 , wherein the schedule indicates particular intervals for the data collection node to send data collection requests to the at least one component of the hosted computing environment.

4. The method of claim 1 , wherein the schedule includes time constraints indicative of a first range of time in which the data collection node sends data collection requests to the at least one component of the hosted computing environment and a second range of time in which the data collection node does not send data collection requests to the at least one component of the hosted computing environment.

5. The method of claim 1 , wherein data collection requests are sent to an application programming interface (API) associated with the hosted computing environment.

6. The method of claim 1 , wherein the instructions for the data collection node to modify the schedule include instructions to increase a rate at which the data collection node sends data collection requests to the at least one component of the hosted computing environment.

7. The method of claim 1 , wherein the instructions for the data collection node to modify the schedule include instructions to decrease a rate at which the data collection node sends data collection requests to the at least one component of the hosted computing environment.

8. The method of claim 1 , wherein the data collection performance information includes information indicating one or more error messages received by the data collection node from the at least one component of the hosted computing environment.

9. The method of claim 1 , wherein the data collection performance information includes network traffic data.

10. The method of claim 1 , wherein the data collection performance information includes CPU utilization information for the data collection node.

11. The method of claim 1 , wherein the data collection performance information includes information indicating one or more attributes of the hosted computing environment.

12. The method of claim 1 , wherein the data collection performance information includes information indicating one or more data collection policies of the hosted computing environment.

13. The method of claim 1 , wherein the controller node receives particular data collection performance information from a plurality of data collection nodes, wherein the plurality of data collection nodes includes the data collection node.

14. The method of claim 1 , wherein the controller node receives particular data collection performance information from a plurality of data collection nodes, wherein the plurality of data collection nodes includes the data collection node, and wherein the plurality of data collection nodes send data collection requests to a plurality of hosted computing environments.

15. The method of claim 1 , wherein the data collection node sends data collection requests to a plurality of hosted computing environments.

16. The method of claim 1 , further comprising:

subsequent to generating the instructions, receiving additional data collection performance information from the data collection node;

generating updated instructions for the data collection node to modify the schedule according to which the data collection node sends data collection requests to the at least one component of the hosted computing environment, wherein the updated instructions to modify the schedule are generated based on the received additional data collection performance information; and

sending the updated instructions to the data collection node.

17. The method of claim 1 , wherein the data collection node sends data collection requests to at least one component of each of a plurality of hosted computing environments, and wherein the controller node generates and sends to the data collection node separate instructions for each different hosted computing environment of the plurality of hosted computing environments.

18. The method of claim 1 , further comprising:

generating second instructions for a different data collection node to send data collection requests to the at least one component of the hosted computing environment, wherein the instructions to modify the schedule are generated based on the received data collection performance information; and

sending the second instructions to the different data collection node.

19. The method of claim 1 , wherein data collection requests sent to the at least one component of the hosted computing environment request data related to performance and/or security information for one or more distributed computing devices of the hosted computing environment.

20. The method of claim 1 , wherein data collected from the hosted computing environment is sent to one or more indexers to be parsed and stored as event data reflecting events of one or more resources of an information technology environment.

21. The method of claim 1 , wherein data collected from the hosted computing environment comprises raw data sent to one or more indexers, and wherein the one or more indexers parse a plurality of timestamped events from the raw data, and wherein each timestamped event of the plurality of timestamped events includes a portion of the raw data.

22. The method of claim 1 , further comprising:

receiving, by the data collection node, the instructions; and

sending, by the data collection node, data collection requests to the at least one component of the hosted computing environment based on the instructions.

23. The method of claim 1 , wherein the data collection node is a first data collection node, the method further comprising:

receiving, by the controller node, an indication that a second data collection node is available; and

modifying instructions for the first data collection node based on the indication that the second data collection node is available.

24. The method of claim 1 , wherein the data collection node is a first data collection node, the method further comprising:

receiving, by the controller node, an indication that the first data collection node is not available; and

modifying instructions for a second data collection node based on the indication that first data collection node is not available.

25. The method of claim 1 , further comprising:

determining that the data collection performance information indicates that the hosted computing environment includes additional data collection capabilities, wherein

based on said determining, the instructions for the data collection node to modify the schedule include instructions to increase a rate at which the data collection node sends data collection requests to the at least one component of the hosted computing environment.

26. The method of claim 1 , wherein the data collection performance information indicates that the hosted computing environment includes additional data collection capabilities based on one or more of: one or more status messages, one or more version identifiers, an operational pattern of distributed computing devices of the hosted computing environment, and data retrieved from a website separate from the hosted computing environment, the method further comprising instructing the data collection node to increase a rate at which the data collection node sends data collection requests to the at least one component of the hosted computing environment based at least in part on the data collection performance information.

27. The method of claim 1 , wherein the instructions to modify the schedule include instructions to change a frequency at which the data collection node sends data collection requests to the at least one component of the hosted computing environment.

28. A controller node, the controller node comprising:

one or more processing devices configured to:

receive data collection performance information from a data collection node, the data collection performance information related to data collection requests sent by the data collection node to at least one component of a plurality of components of a hosted computing environment, the plurality of components of the hosted computing environment including distributed computing devices used to generate computing environments that share compute resources of a particular distributed computing device of the distributed computing devices;

generate instructions for the data collection node to modify a schedule according to which the data collection node sends data collection requests to the at least one component of the hosted computing environment, wherein the instructions to modify the schedule are generated based on the received data collection performance information; and

send the instructions to the data collection node.

29. The controller node of claim 28 , wherein the schedule includes time constraints indicative of a first range of time in which the data collection node sends data collection requests to the at least one component of the hosted computing environment and a second range of time in which the data collection node does not send data collection requests to the at least one component of the hosted computing environment.

30. Non-transitory computer readable media comprising computer-executable instructions that, when executed by a controller node, cause the controller node to:

receive data collection performance information from a data collection node, the data collection performance information related to data collection requests sent by the data collection node to at least one component of a plurality of components of a hosted computing environment, the plurality of components of the hosted computing environment including distributed computing devices used to generate computing environments that share compute resources of a particular distributed computing device of the distributed computing devices;

generate instructions for the data collection node to modify a schedule according to which the data collection node sends data collection requests to the at least one component of the hosted computing environment, wherein the instructions to modify the schedule are generated based on the received data collection performance information; and

send the instructions to the data collection node.

Assignments (3)
CHANGE OF NAME Recorded Jul 22, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 072170/0599 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2025
From: SPLUNK LLC
To: CISCO TECHNOLOGY, INC.
Reel/Frame 072173/0058 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 31, 2018
From: CHEN, KEN; TAO, GANG; DING, LAI QIANG; HAO, JUNQING; WANG, TING; HADDAD, ELIAS; BITINCKA, DRITAN
To: SPLUNK INC.
Reel/Frame 045955/0987 →
Continuity (3)
Continuation 15011525 · Jan 30, 2016
Continuation In Part 14902848
Related Publication 20180246942A1 · Aug 30, 2018