Anomaly-based on-demand collection of data by an agent for a data platform
An illustrative data platform may receive, from an agent configuration deployed in a cloud environment and configured to monitor compute assets in the cloud environment, first data periodically collected by the agent configuration at a first collection frequency and second data periodically collected by the agent configuration at a second collection frequency, identify, based on the first data, an anomaly associated with one or more compute assets included in the compute assets, and direct, based on the identifying the anomaly, the agent configuration to perform an on-demand collection of the second data.
1 . A method comprising:
receiving, by a data platform from an agent configuration deployed in a cloud environment and configured to monitor compute assets in the cloud environment, first data periodically collected by the agent configuration at a first collection frequency and second data periodically collected by the agent configuration at a second collection frequency;
identifying, by the data platform based on an analysis of the first data, an anomaly associated with one or more compute assets included in the compute assets; and
transmitting, by the data platform in response to the identifying the anomaly based on the analysis of the first data, configuration data to the agent configuration, the configuration data including one or more commands that specify one or more parameters associated with at least one of the first data and the second data, wherein at least one of the one or more parameters specified by the one or more commands directs the agent configuration to perform an on-demand collection of the second data.
2 . The method of claim 1 , wherein:
the agent configuration comprises a first agent configured to collect the first data and a second agent configured to collect the second data, and wherein the at least one of the one or more parameters specified by the one or more commands directs the second agent configuration to perform the on-demand collection of the second data.
3 . The method of claim 2 , wherein:
the first agent is deployed within a node within the cloud environment;
the first data comprises data associated with one or more compute assets within the node;
the second agent is deployed within a cluster of which the node is a part; and
the second data comprises data associated with the cluster.
4 . The method of claim 2 , wherein:
the first agent is deployed within a cluster within the cloud environment and comprising a plurality of nodes;
the first data comprises data associated with the cluster;
the second agent is deployed within a node of the cluster;
the second data comprises data associated with one or more compute assets within the node.
5 . The method of claim 2 , wherein the identifying the anomaly comprises receiving, from the first agent, a request provided by the first agent requesting that the second agent perform the on-demand collection of the second data.
6 . The method of claim 1 , wherein the agent configuration comprises a single agent configured to collect both the first and second data.
7 . The method of claim 1 , wherein the on-demand collection occurs at a time different than when the periodic collection of the second data occurs.
8 . The method of claim 1 , further comprising:
receiving, by the data platform, the second data collected by the on-demand collection;
and performing, by the data platform based on the second data, an operation with respect to the one or more compute assets.
9 . The method of claim 8 , wherein the performing the operation comprises one or more of generating an alert, determining a compliance level of the one or more compute assets, determining that the one or more compute assets are subject to a security threat, modifying one or both of the first collection frequency or the second collection frequency, providing a recommendation with respect to the one or more compute assets, or performing a remedial action with respect to the one or more compute assets.
10 . The method of claim 1 , further comprising:
constructing, by the data platform based on the first data, a graph comprising a plurality of nodes connected by a plurality of edges, wherein each node of the plurality of nodes represents a logical entity and each edge of the plurality of edges represents a behavioral relationship between nodes connected by the edge;
wherein the identifying the anomaly is based on the graph.
11 . The method of claim 1 , further comprising:
applying the first data to a machine learning model; and
the identifying the anomaly is based on an output of the machine learning model.
12 . The method of claim 1 , wherein:
the second data is relatively more static than the first data; and
the first collection frequency is greater than the second collection frequency.
13 . A system comprising:
a memory storing instructions; and
one or more processors communicatively coupled to the memory and configured to execute the instructions to perform a process comprising:
receiving, from an agent configuration deployed in a cloud environment and configured to monitor compute assets in the cloud environment, first data periodically collected by the agent configuration at a first collection frequency and second data periodically collected by the agent configuration at a second collection frequency;
identifying, based on an analysis of the first data, an anomaly associated with one or more compute assets included in the compute assets; and
transmitting, in response to the identifying the anomaly based on the analysis of the first data, configuration data to the agent configuration, the configuration data including one or more commands that specify one or more parameters associated with at least one of the first data and the second data, wherein at least one of the one or more parameters specified by the one or more commands directs the agent configuration to perform an on-demand collection of the second data.
14 . The system of claim 13 , wherein:
the agent configuration comprises a first agent configured to collect the first data and a second agent configured to collect the second data, and wherein the at least one of the one or more parameters specified by the one or more commands directs the second agent configuration to perform the on-demand collection of the second data.
15 . The system of claim 14 , wherein:
the first agent is deployed within a node within the cloud environment;
the first data comprises data associated with one or more compute assets within the node;
the second agent is deployed within a cluster of which the node is a part; and
the second data comprises data associated with the cluster.
16 . The system of claim 14 , wherein:
the first agent is deployed within a cluster within the cloud environment and comprising a plurality of nodes;
the first data comprises data associated with the cluster;
the second agent is deployed within a node of the cluster;
the second data comprises data associated with one or more compute assets within the node.
17 . The system of claim 14 , wherein the identifying the anomaly comprises receiving, from the first agent, a request provided by the first agent requesting that the second agent perform the on-demand collection of the second data.
18 . The system of claim 13 , wherein the agent configuration comprises a single agent configured to collect both the first and second data.
19 . The system of claim 13 , wherein the process further comprises:
receiving the second data collected by the on-demand collection; and
performing, based on the second data, an operation with respect to the one or more compute assets.
20 . A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving, from an agent configuration deployed in a cloud environment and configured to monitor compute assets in the cloud environment, first data periodically collected by the agent configuration at a first collection frequency and second data periodically collected by the agent configuration at a second collection frequency;
identifying, based on an analysis of the first data, an anomaly associated with one or more compute assets included in the compute assets; and
transmitting, in response to the identifying the anomaly based on the analysis of the first data, one or more commands that specify one or more parameters associated with at least one of the first data and the second data to the agent configuration, wherein at least one of the one or more parameters specified by the one or more commands directs the agent configuration to perform an on-demand collection of the second data.