Notification-based file integrity monitoring of a compute environment
Data platforms and agents described herein are configured to intercommunicate to perform security monitoring of a compute environment by way of notification-based file integrity monitoring. As described herein, an agent may identify a file designated for file integrity monitoring, register to be notified by an operating system of the compute environment when a change to the file occurs, perform a file integrity monitoring operation based on a notification received in connection with the registering, and communicate data associated with the file integrity monitoring operation to the data platform in real time. The data platform may designate the file within the compute environment for file integrity monitoring, direct the agent to register to be notified by the operating system when a change to the file occurs, and receive the data associated with the file integrity monitoring operation. Corresponding methods, systems, and products are also disclosed.
1 . A method comprising:
identifying, by an agent deployed to a compute environment and configured to communicate with a data platform performing security monitoring of the compute environment, a file designated for file integrity monitoring;
constructing a graph for data received and ingested via the agent, wherein the graph has at least data associated with file integrity monitoring operations performed by the agent;
registering, by the agent, to be notified by an operating system of the compute environment when a change to the file occurs;
performing, by the agent and based on a notification received in connection with the registering, a file integrity monitoring operation, wherein the notification is included among a plurality of notifications that are received in connection with the registering and indicate different changes to the file; and
the file integrity monitoring operation includes:
determining that a first change to the file indicated by the notification occurs within a threshold amount of time to a second change to the file indicated by an additional notification of the plurality of notifications, and
based on the determining, aggregating information about the first change and the second change within the data communicated to the data platform; and
communicating, by the agent to the data platform in real time, data associated with the file integrity monitoring operation comprising at least a portion of the graph.
2 . The method of claim 1 , wherein the file integrity monitoring operation includes:
determining a first checksum for the file, the first checksum associated with a point in time after the change to the file occurs; and
at least one of:
comparing the first checksum with a second checksum for the file, the second checksum associated with a point in time before the change to the file occurs, or
comparing the first checksum with a third checksum for the file, the third checksum associated with a known malicious file.
3 . The method of claim 1 , wherein the threshold amount of time is a configurable amount of time that has a default value adjustable by way of input to the data platform.
4 . The method of claim 1 , wherein the file integrity monitoring operation includes analyzing the change to the file to determine one or more properties from a set of properties including:
a file type of the file that has changed;
an entity responsible for the change to the file; and
an indicator of how the file is changed.
5 . The method of claim 4 , wherein:
the notification received in connection with the registering is included among a plurality of notifications that are received in connection with the registering and indicate different changes to the file; and
the file integrity monitoring operation includes filtering the different changes to the file based the set of properties.
6 . The method of claim 1 , wherein the file designated for file integrity monitoring is managed by a filesystem of a container running on top of the operating system.
7 . The method of claim 1 , wherein:
the file identified to be designated for file integrity monitoring includes sensitive information that is restricted by a set of rules; and
the performing of the file integrity monitoring operation and the communicating of the data associated with the file integrity monitoring operation are performed in accordance with the set of rules.
8 . The method of claim 1 , wherein:
a user interface provided by the data platform supports manual designation of individual files, directories of files, and configurable subdirectories of files for file integrity monitoring; and
the identified file is designated for file integrity monitoring manually by way of the user interface.
9 . The method of claim 1 , further comprising tracking, by the agent, usage statistics for a plurality of files within the compute environment, the plurality of files including the file;
wherein the identified file is designated for file integrity monitoring automatically based on the tracked usage statistics.
10 . The method of claim 1 , wherein:
the operating system of the compute environment is a Linux-based operating system; and
the notification received in connection with the registering is received by way of a kernel application programming interface (API) supported by the Linux-based operating system.
11 . The method of claim 1 , wherein:
the operating system of the compute environment is a Linux-based operating system; and the notification received in connection with the registering is received by way of a notification service with which the agent has registered, the notification service deployed to a kernel of the Linux-based operating system using extended Berkeley Packet Filter (eBPF) technology.
12 . The method of claim 1 , further comprising:
tracking, by the data platform based on the data associated with the file integrity monitoring operation and communicated by the agent, activity related to the file to detect a malicious file spread; and
generating, by the data platform based on the tracking, an alert indicative of the malicious file spread.
13 . The method of claim 1 , further comprising:
constructing and updating, by the data platform based on the data associated with the file integrity monitoring operation and communicated by the agent, a graph comprising a plurality of nodes connected by a plurality of edges, wherein each node of the plurality of nodes represents a logical entity and each edge of the plurality of edges represents a behavioral relationship between nodes connected by the edge; and
generating, by the data platform based on the graph, an alert indicative of an anomaly or security threat detected within the compute environment.
14 . A computer program product embodied in a non-transitory computer-readable storage medium and comprising computer instructions for an agent deployed to a compute environment to perform a process comprising:
identifying a file designated for file integrity monitoring within the compute environment;
constructing a graph for data received and ingested via the agent, wherein the graph has at least data associated with file integrity monitoring operations performed by the agent;
registering to be notified by an operating system of the compute environment when a change to the file occurs;
performing, based on a notification received in connection with the registering, a file integrity monitoring operation, wherein the notification is included among a plurality of notifications that are received in connection with the registering and indicate different changes to the file; and
the file integrity monitoring operation includes:
determining that a first change to the file indicated by the notification occurs within a threshold amount of time to a second change to the file indicated by an additional notification of the plurality of notifications, and
based on the determining, aggregating information about the first change and the second change within the data communicated to the data platform; and
communicating, in real time to a data platform performing security monitoring of the compute environment, data associated with the file integrity monitoring operation comprising at least a portion of the graph.
15 . The computer program product of claim 14 , wherein the file integrity monitoring operation includes:
determining a first checksum for the file, the first checksum associated with a point in time after the change to the file occurs; and
at least one of:
comparing the first checksum with a second checksum for the file, the second checksum associated with a point in time before the change to the file occurs, or
comparing the first checksum with a third checksum for the file, the third checksum associated with a known malicious file.
16 . The computer program product of claim 14 , wherein the file integrity monitoring operation includes analyzing the change to the file to determine one or more properties from a set of properties including:
a file type of the file that has changed;
an entity responsible for the change to the file; and
an indicator of how the file is changed.
17 . The computer program product of claim 14 , wherein the file designated for file integrity monitoring is managed by a filesystem of a container running on top of the operating system.
18 . A method for performing security monitoring of a compute environment, the method comprising:
designating, by a data platform, a file within the compute environment for file integrity monitoring;
directing, by the data platform and by way of communication with an agent deployed to the compute environment, the agent to register to be notified by an operating system of the compute environment when a change to the file occurs;
constructing a graph for data received and ingested via the agent, wherein the graph has incorporated within it at least data associated with file integrity monitoring operations performed by the agent; and
receiving, by the data platform and in real time subsequent to a file integrity monitoring operation being performed by the agent based on a notification received in connection with the agent registering to be notified, data associated with the file integrity monitoring operation, wherein the notification is included among a plurality of notifications that are received in connection with the registering and indicate different changes to the file; and
the file integrity monitoring operation includes:
determining that a first change to the file indicated by the notification occurs within a threshold amount of time to a second change to the file indicated by an additional notification of the plurality of notifications, and
based on the determining, aggregating information about the first change and the second change within the data communicated to the data platform.