Configuring rules for filtering events to be included in event streams
The disclosed embodiments provide a system that processes network data. During operation, the system obtains, at a remote capture agent, a first protocol classification for a first packet flow captured by the remote capture agent. Next, the system uses configuration information associated with the first protocol classification to build a first event stream from the first packet flow at the remote capture agent, wherein the first event stream comprises time-series event data generated from network packets in the first packet flow based on the first protocol classification. The system then transmits the first event stream over a network for subsequent storage and processing of the first event stream by one or more components on the network.
1. A computer-implemented method performed by a configuration server coupled to a network, the method comprising:
receiving input defining a filter to be applied to timestamped events generated by one or more remote capture agents from network data monitored by the one or more remote capture agents, the filter used to identify a subset of the timestamped events to be included in one or more event streams generated by the one or more remote capture agents;
generating configuration data based at least in part on the input defining the filter; and
sending, over the network, the configuration data to the one or more remote capture agents, the configuration data causing the one or more remote capture agents to apply the filter to events generated by the one or more remote capture agents for inclusion in the one or more event streams.
2. The computer-implemented method of claim 1 , further comprising:
obtaining, by the one or more remote capture agents, the configuration data from the configuration server; and
using the configuration data to configure generation of the one or more event streams.
3. The computer-implemented method of claim 1 , wherein sending the configuration data to the one or more remote capture agents causes the one or more remote capture agent to configure generation of the timestamped events from the network data during runtime of the one or more remote capture agents.
4. The computer-implemented method of claim 1 , wherein the input is first input, and wherein the method further comprises:
receiving second input indicating a modification to the filter;
generating updated configuration data based at least in part on the second input; and
sending the updated configuration data to the one or more remote capture agents.
5. The computer-implemented method of claim 1 , wherein the configuration data instructs the one or more remote capture agents to send the one or more event streams to another component on the network for subsequent processing.
6. The computer-implemented method of claim 1 , wherein the configuration data instructs the one or more remote capture agents to, in response to detecting encryption of the network data, decrypt the network data prior to generating the one or more event streams.
7. The computer-implemented method of claim 1 , wherein the one or more remote capture agents generate the one or more event streams in part by:
identifying one or more event attributes defined in the configuration data;
extracting the one or more event attributes from network packets comprising the network data; and
including the one or more event attributes in the one or more event streams.
8. The computer-implemented method of claim 1 , wherein the one or more remote capture agents generate the one or more event streams in part by:
identifying one or more event attributes defined in the configuration data;
extracting the one or more event attributes from network packets comprising the network data;
transforming, based on the configuration data, the one or more event attributes extracted from the network packets to obtain one or more transformed event attributes; and
including the one or more transformed event attributes in the one or more event streams.
9. The computer-implemented method of claim 1 , wherein the input is received via a graphical user interface (GUI) including interface elements that enable input defining the filter.
10. The computer-implemented method of claim 1 , wherein at least one of the one or more remote capture agents is installed in a cloud computing environment.
11. A configuration server, comprising:
a processor;
a non-transitory computer readable storage medium storing instructions which, when executed by the processor, cause the configuration server to:
receive input defining a filter to be applied to timestamped events generated by one or more remote capture agents from network data monitored by the one or more remote capture agents, the filter used to identify a subset of the timestamped events to be included in one or more event streams generated by the one or more remote capture agents;
generate configuration data based at least in part on the input defining the filter; and
send, over a network, the configuration data to the one or more remote capture agents, the configuration data causing the one or more remote capture agents to apply the filter to events generated by the one or more remote capture agents for inclusion in the one or more event streams.
12. The configuration server of claim 11 , wherein the instructions, when executed by the processor, further cause the configuration server to:
cause the one or more remote capture agents to obtain the configuration data from the configuration server; and
cause the one or more remote capture agents to use the configuration data to configure generation of the one or more event streams.
13. The configuration server of claim 11 , wherein sending the configuration data to the one or more remote capture agents causes the one or more remote capture agent to configure generation of the timestamped events from the network data during runtime of the one or more remote capture agents.
14. The configuration server of claim 11 , wherein the input is first input, and wherein the instructions, when executed by the processor, further cause the configuration to:
receive second input indicating a modification to the filter;
generate updated configuration data based at least in part on the second input; and
send the updated configuration data to the one or more remote capture agents.
15. The configuration server of claim 11 , wherein the configuration data instructs the one or more remote capture agents to send the one or more event streams to another component on the network for subsequent processing.
16. The configuration server of claim 11 , wherein the configuration data instructs the one or more remote capture agents to, in response to detecting encryption of the network data, decrypt the network data prior to generating the one or more event streams.
17. The configuration server of claim 11 , wherein the one or more remote capture agents generate the one or more event streams in part by:
identifying one or more event attributes defined in the configuration data;
extracting the one or more event attributes from network packets comprising the network data; and
including the one or more event attributes in the one or more event streams.
18. The configuration server of claim 11 , wherein the one or more remote capture agents generate the one or more event streams in part by:
identify one or more event attributes defined in the configuration data;
extract the one or more event attributes from network packets comprising the network data;
transform, based on the configuration data, the one or more event attributes extracted from the network packets to obtain one or more transformed event attributes; and
include the one or more transformed event attributes in the one or more event streams.
19. The configuration server of claim 11 , wherein the input is received via a graphical user interface (GUI) including interface elements that enable input defining the filter.
20. The configuration server of claim 11 , wherein at least one of the one or more remote capture agents is installed in a cloud computing environment.
21. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform operations comprising:
receiving, by a configuration server, input defining a filter to be applied to timestamped events generated by one or more remote capture agents from network data monitored by the one or more remote capture agents, the filter used to identify a subset of the timestamped events to be included in one or more event streams generated by the one or more remote capture agents;
generating configuration data based at least in part on the input defining the filter; and
sending, over a network, the configuration data to the one or more remote capture agents, the configuration data causing the one or more remote capture agents to apply the filter to events generated by the one or more remote capture agents for inclusion in the one or more event streams.
22. The non-transitory computer-readable storage medium of claim 21 , wherein the instructions, when executed, further cause the configuration server to:
cause the one or more remote capture agents to obtain the configuration data from the configuration server; and
cause the one or more remote capture agents to use the configuration data to configure generation of the one or more event streams.
23. The non-transitory computer-readable storage medium of claim 21 , wherein sending the configuration data to the one or more remote capture agents causes the one or more remote capture agent to configure generation of the timestamped events from the network data during runtime of the one or more remote capture agents.
24. The non-transitory computer-readable storage medium of claim 21 , wherein the input is first input, and wherein the instructions, when executed, further cause the configuration server to:
receive second input indicating a modification to the filter;
generate updated configuration data based at least in part on the second input; and
send the updated configuration data to the one or more remote capture agents.
25. The non-transitory computer-readable storage medium of claim 21 , wherein the configuration data instructs the one or more remote capture agents to send the one or more event streams to another component on the network for subsequent processing.
26. The non-transitory computer-readable storage medium of claim 21 , wherein the configuration data instructs the one or more remote capture agents to, in response to detecting encryption of the network data, decrypt the network data prior to generating the one or more event streams.
27. The non-transitory computer-readable storage medium of claim 21 , wherein the one or more remote capture agents generate the one or more event streams in part by:
identifying one or more event attributes defined in the configuration data;
extracting the one or more event attributes from network packets comprising the network data; and
including the one or more event attributes in the one or more event streams.
28. The non-transitory computer-readable storage medium of claim 21 , wherein the one or more remote capture agents generate the one or more event streams in part by:
identifying one or more event attributes defined in the configuration data;
extracting the one or more event attributes from network packets comprising the network data;
transforming, based on the configuration data, the one or more event attributes extracted from the network packets to obtain one or more transformed event attributes; and
including the one or more transformed event attributes in the one or more event streams.
29. The non-transitory computer-readable storage medium of claim 21 , wherein the input is received via a graphical user interface (GUI) including interface elements that enable input defining the filter.
30. The non-transitory computer-readable storage medium of claim 21 , wherein at least one of the one or more remote capture agents is installed in a cloud computing environment.