Selective event stream data storage based on network traffic volume
The disclosed embodiments provide a system that facilitates the processing of network data. During operation, the system causes for display a graphical user interface (GUI) for configuring the generation of time-series event data from network packets captured by one or more remote capture agents. Next, the system causes for display, in the GUI, a first set of user-interface elements containing a set of statistics associated with one or more event streams that comprise the time-series event data. The system then causes for display, in the GUI, one or more graphs comprising one or more values from the set of statistics. Finally, the system causes for display, in the GUI, a value of a statistic from the set of statistics based on a position of a cursor over the one or more graphs.
1. A computer-implemented method, comprising:
receiving an event stream from a remote capture agent, wherein the event stream includes timestamped event data generated by the remote capture agent based on network traffic monitored by the remote capture agent;
calculating, based on historical data over one or more periods of time, an unused amount of storage relative to a storage limit associated with the event stream;
determining, based on the unused amount of storage, a suggested percentage of the event stream to store in one or more data stores;
causing display of a graphical user interface (GUI) including text indicating the suggested percentage of the event stream to store in the one or more data stores;
receiving input of a specified percentage of the event stream to store in the one or more data stores; and
causing a portion of the event stream to be stored in the one or more data stores according to the specified percentage.
2. The method of claim 1 , further comprising causing display, in the GUI, of an interface element for adjusting the specified percentage of the event stream to store in the one or more data stores.
3. The method of claim 2 , wherein receiving input of a specified percentage of the event stream to store in the one or more data stores comprises receiving input of movement of the interface element within the GUI.
4. The method of claim 2 , wherein the interface element is a slider.
5. The method of claim 1 , wherein the portion of the event stream is a first portion, and wherein causing the first portion of the event stream to be stored in the one or more data stores includes discarding a second portion of the event stream.
6. The method of claim 1 , wherein a volume of the network traffic from which the remote capture agent generated the timestamped event data is a first volume of first network traffic as measured at a first point in time, wherein the suggested percentage of the event stream to store in one or more data stores is a first suggested percentage, and wherein the method further comprises:
determining a second volume of second network traffic from which the remote capture agent generates timestamped event data as measured at a second point in time, wherein the second volume of second network traffic is greater than the first volume of first network traffic;
determining a second suggested percentage of the event stream to store in the one or more data stores based on the second volume of the second network traffic, wherein the second suggested percentage is less than the first suggested percentage; and
causing display of a GUI including text indicating the second suggested percentage of the event stream to store in the one or more data stores.
7. The method of claim 1 , wherein a volume of the network traffic from which the remote capture agent generated the timestamped event data is a first volume of first network traffic as measured at a first point in time, wherein the suggested percentage of the event stream to store in one or more data stores is a first suggested percentage, and wherein the method further comprises:
determining a second volume of second network traffic from which the remote capture agent generates timestamped event data as measured at a second point in time, wherein the second volume of second network traffic is less than the first volume of first network traffic;
determining a second suggested percentage of the event stream to store in the one or more data stores based on the second volume of the second network traffic, wherein the second suggested percentage is greater than the first suggested percentage; and
causing display of a GUI including text indicating the second suggested percentage of the event stream to store in the one or more data stores.
8. A computing device, comprising:
one or more processors; and
a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the computing device to perform operations including:
receiving an event stream from a remote capture agent, wherein the event stream includes timestamped event data generated by the remote capture agent based on network traffic monitored by the remote capture agent;
calculating, based on historical data over one or more periods of time, an unused amount of storage relative to a storage limit associated with the event stream;
determining, based on the unused amount of storage, a suggested percentage of the event stream to store in one or more data stores;
causing display of a graphical user interface (GUI) including text indicating the suggested percentage of the event stream to store in the one or more data stores;
receiving input of a specified percentage of the event stream to store in the one or more data stores; and
causing a portion of the event stream to be stored in the one or more data stores according to the specified percentage.
9. The computing device of claim 8 , wherein the instructions include further instructions that, when executed by the one or more processors, cause the computing device to perform further operations comprising causing display, in the GUI, of an interface element for adjusting the specified percentage of the event stream to store in the one or more data stores.
10. The computing device of claim 9 , wherein receiving input of a specified percentage of the event stream to store in the one or more data stores comprises receiving input of movement of the interface element within the GUI.
11. The computing device of claim 9 , wherein the interface element is a slider.
12. The computing device of claim 8 , wherein the portion of the event stream is a first portion, and wherein causing the first portion of the event stream to be stored in the one or more data stores includes discarding a second portion of the event stream.
13. The computing device of claim 8 , wherein a volume of the network traffic from which the remote capture agent generated the timestamped event data is a first volume of first network traffic as measured at a first point in time, wherein the suggested percentage of the event stream to store in one or more data stores is a first suggested percentage, and wherein the instructions include further instructions that, when executed by the one or more processors, cause the computing device to perform further operations comprising:
determining a second volume of second network traffic from which the remote capture agent generates timestamped event data as measured at a second point in time, wherein the second volume of second network traffic is greater than the first volume of first network traffic;
determining a second suggested percentage of the event stream to store in the one or more data stores based on the second volume of the second network traffic, wherein the second suggested percentage is less than the first suggested percentage; and
causing display of a GUI including text indicating the second suggested percentage of the event stream to store in the one or more data stores.
14. The computing device of claim 8 , wherein a volume of the network traffic from which the remote capture agent generated the timestamped event data is a first volume of first network traffic as measured at a first point in time, wherein the suggested percentage of the event stream to store in one or more data stores is a first suggested percentage, and wherein the instructions include further instructions that, when executed by the one or more processors, cause the computing device to perform further operations comprising:
determining a second volume of second network traffic from which the remote capture agent generates timestamped event data as measured at a second point in time, wherein the second volume of second network traffic is less than the first volume of first network traffic;
determining a second suggested percentage of the event stream to store in the one or more data stores based on the second volume of the second network traffic, wherein the second suggested percentage is greater than the first suggested percentage; and
causing display of a GUI including text indicating the second suggested percentage of the event stream to store in the one or more data stores.
15. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including:
receiving an event stream from a remote capture agent, wherein the event stream includes timestamped event data generated by the remote capture agent based on network traffic monitored by the remote capture agent;
calculating, based on historical data over one or more periods of time, an unused amount of storage relative to a storage limit associated with the event stream;
determining, based on the unused amount of storage, a suggested percentage of the event stream to store in one or more data stores;
causing display of a graphical user interface (GUI) including text indicating the suggested percentage of the event stream to store in the one or more data stores;
receiving input of a specified percentage of the event stream to store in the one or more data stores; and
causing a portion of the event stream to be stored in the one or more data stores according to the specified percentage.
16. The non-transitory computer-readable medium of claim 15 , wherein the instructions include further instructions that, when executed by the one or more processors, cause the computing device to perform further operations comprising causing display, in the GUI, of an interface element for adjusting the specified percentage of the event stream to store in the one or more data stores.
17. The non-transitory computer-readable medium of claim 16 , wherein receiving input of a specified percentage of the event stream to store in the one or more data stores comprises receiving input of movement of the interface element within the GUI.
18. The non-transitory computer-readable medium of claim 15 , wherein the portion of the event stream is a first portion, and wherein causing the first portion of the event stream to be stored in the one or more data stores includes discarding a second portion of the event stream.
19. The non-transitory computer-readable medium of claim 15 , wherein a volume of the network traffic from which the remote capture agent generated the timestamped event data is a first volume of first network traffic as measured at a first point in time, wherein the suggested percentage of the event stream to store in one or more data stores is a first suggested percentage, and wherein the instructions include further instructions that, when executed by the one or more processors, cause the computing device to perform further operations comprising:
determining a second volume of second network traffic from which the remote capture agent generates timestamped event data as measured at a second point in time, wherein the second volume of second network traffic is greater than the first volume of first network traffic;
determining a second suggested percentage of the event stream to store in the one or more data stores based on the second volume of the second network traffic, wherein the second suggested percentage is less than the first suggested percentage; and
causing display of a GUI including text indicating the second suggested percentage of the event stream to store in the one or more data stores.
20. The non-transitory computer-readable medium of claim 15 , wherein a volume of the network traffic from which the remote capture agent generated the timestamped event data is a first volume of first network traffic as measured at a first point in time, wherein the suggested percentage of the event stream to store in one or more data stores is a first suggested percentage, and wherein the instructions include further instructions that, when executed by the one or more processors, cause the computing device to perform further operations comprising:
determining a second volume of second network traffic from which the remote capture agent generates timestamped event data as measured at a second point in time, wherein the second volume of second network traffic is less than the first volume of first network traffic;
determining a second suggested percentage of the event stream to store in the one or more data stores based on the second volume of the second network traffic, wherein the second suggested percentage is greater than the first suggested percentage; and
causing display of a GUI including text indicating the second suggested percentage of the event stream to store in the one or more data stores.