Interval-based generation of event streams by remote capture agents
The disclosed embodiments provide a system that facilitates the processing of network data. During operation, the system obtains a set of event streams from one or more remote capture agents over one or more networks, wherein the set of event streams comprises time-series event data generated from network packets captured by the one or more remote capture agents. Next, the system causes for display, within a graphical user interface (GUI), a first set of user interface elements, wherein the first set of user interface elements includes event stream information for an event stream in the set of event streams and a first graph of a metric associated with the time-series event data in the event stream. The system then updates the first graph in real-time with the time-series event data from the one or more remote capture agents.
1. A computer-implemented method performed by a remote capture agent located in a computer network, the method comprising:
obtaining configuration information used by the remote capture agent to generate an event stream, wherein the event stream includes a plurality of timestamped events generated based on network data monitored by the remote capture agent, and wherein the configuration information specifies an interval at which the remote capture agent generates timestamped events of the event stream;
monitoring network data sent or received by a computing device located in the computer network;
generating, according to the interval specified in the configuration information, the plurality of timestamped events based on the network data; and
sending the plurality of timestamped events to another component on the network.
2. The method of claim 1 , wherein the interval specifies an amount of time between generation of consecutive timestamped events by the remote capture agent.
3. The method of claim 1 , wherein the interval specifies an amount of time between generation of consecutive timestamped events, and wherein generating a timestamped event of the plurality of timestamped events includes aggregating data included in the network data during an interval of time associated with the timestamped event.
4. The method of claim 1 , wherein the configuration information specifies an aggregation attribute identifying a type of data derivable from the network data monitored by the remote capture agent, and wherein generating a timestamped event of the plurality of timestamped events includes aggregating data associated with the aggregation attribute obtained during an interval of time associated with the timestamped event.
5. The method of claim 1 , wherein the configuration information specifies an aggregation attribute identifying a type of data derivable from the network data monitored by the remote capture agent, wherein the configuration information further specifies an aggregation statistic to obtain based on the aggregation attribute, wherein the aggregation statistic is at least one of: a sum, an average, a maximum, a minimum, or a standard deviation, and wherein generating a timestamped event of the plurality of timestamped events includes generating the aggregation statistic based on data associated with the aggregation attribute obtained during an interval of time associated with the timestamped event.
6. The method of claim 1 , further comprising displaying data associated with the event stream in a graphical user interface.
7. The method of claim 1 , wherein the configuration information specifies data to be included in the plurality of events based on a network protocol associated with the network data, and wherein the method further comprises:
determining that the network protocol is used by the network data; and
wherein generating a timestamped event of the plurality of timestamped events includes obtaining protocol-specific data from the network data.
8. The method of claim 1 , wherein the configuration information specifies a custom field to be included in timestamped events of the plurality of timestamped events, and wherein generating a timestamped event of the plurality of timestamped events includes generating a value for the custom field and including the value in the timestamped event.
9. The method of claim 1 , wherein the configuration information specifies a transformation to be applied to timestamped events of the plurality of timestamped events, and wherein generating a timestamped event of the plurality of timestamped events includes applying the transformation to data included in the timestamped event.
10. The method of claim 1 , wherein the configuration information indicates that the event stream is an ephemeral event stream associated with an end time, and wherein generating the plurality of timestamped events includes terminating generation of the plurality of timestamped events at the end time.
11. An apparatus, comprising:
one or more hardware processors; and
memory storing instructions that, when executed by the one or more hardware processors, cause the apparatus to:
obtain configuration information used by a remote capture agent to generate an event stream, wherein the event stream includes a plurality of timestamped events generated based on network data monitored by the remote capture agent, and wherein the configuration information specifies an interval at which the remote capture agent generates timestamped events of the event stream;
monitor network data sent or received by a computing device located in the computer network;
generate, according to the interval specified in the configuration information, the plurality of timestamped events based on the network data; and
send the plurality of timestamped events to another component on the network.
12. The apparatus of claim 11 , wherein the interval specifies an amount of time between generation of consecutive timestamped events by the remote capture agent.
13. The apparatus of claim 11 , wherein the interval specifies an amount of time between generation of consecutive timestamped events, and wherein generating a timestamped event of the plurality of timestamped events includes aggregating data included in the network data during an interval of time associated with the timestamped event.
14. The apparatus of claim 11 , wherein the configuration information specifies an aggregation attribute identifying a type of data derivable from the network data monitored by the remote capture agent, and wherein generating a timestamped event of the plurality of timestamped events includes aggregating data associated with the aggregation attribute obtained during an interval of time associated with the timestamped event.
15. The apparatus of claim 11 , wherein the configuration information specifies an aggregation attribute identifying a type of data derivable from the network data monitored by the remote capture agent, wherein the configuration information further specifies an aggregation statistic to obtain based on the aggregation attribute, wherein the aggregation statistic is at least one of: a sum, an average, a maximum, a minimum, or a standard deviation, and wherein generating a timestamped event of the plurality of timestamped events includes generating the aggregation statistic based on data associated with the aggregation attribute obtained during an interval of time associated with the timestamped event.
16. A non-transitory computer-readable storage medium storing instructions which, when executed by one or more hardware processors, cause performance of operations comprising:
obtaining configuration information used by a remote capture agent located in a computer network to generate an event stream, wherein the event stream includes a plurality of timestamped events generated based on network data monitored by the remote capture agent, and wherein the configuration information specifies an interval at which the remote capture agent generates timestamped events of the event stream;
monitoring network data sent or received by a computing device located in the computer network;
generating, according to the interval specified in the configuration information, the plurality of timestamped events based on the network data; and
sending the plurality of timestamped events to another component on the network.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the interval specifies an amount of time between generation of consecutive timestamped events by the remote capture agent.
18. The non-transitory computer-readable storage medium of claim 16 , wherein the interval specifies an amount of time between generation of consecutive timestamped events, and wherein generating a timestamped event of the plurality of timestamped events includes aggregating data included in the network data during an interval of time associated with the timestamped event.
19. The non-transitory computer-readable storage medium of claim 16 , wherein the configuration information specifies an aggregation attribute identifying a type of data derivable from the network data monitored by the remote capture agent, and wherein generating a timestamped event of the plurality of timestamped events includes aggregating data associated with the aggregation attribute obtained during an interval of time associated with the timestamped event.
20. The non-transitory computer-readable storage medium of claim 16 , wherein the configuration information specifies an aggregation attribute identifying a type of data derivable from the network data monitored by the remote capture agent, wherein the configuration information further specifies an aggregation statistic to obtain based on the aggregation attribute, wherein the aggregation statistic is at least one of: a sum, an average, a maximum, a minimum, or a standard deviation, and wherein generating a timestamped event of the plurality of timestamped events includes generating the aggregation statistic based on data associated with the aggregation attribute obtained during an interval of time associated with the timestamped event.