Filtering sensitive data in cloud native application logs
A computer-implemented method to limit access to sensitive information by filtering log files. The method includes deploying a first pod on a node of a cloud computing system, where the first pod includes a first container configured to run an application. The method also includes generating a first log file for the first container, where the first log file includes a set of actions performed by the application for a period of time. The method further includes filtering, by a filter, the first log file wherein the filter is configured to remove a type of sensitive data from the first log file. The method includes exporting, in response to the filtering, the first log file to the node. Advantageously, this can prevent various parties from accessing sensitive data that is contained in log files.
1 . A computer-implemented method comprising:
deploying a first pod on a node of a cloud computing system, wherein the first pod includes a first container configured to run an application and the node is contained within a computing device;
deploying a second pod on the node within the computing device;
generating a first log file for the first container, wherein the first log file includes a set of actions performed by the application for a period of time;
filtering, by a filter, the first log file wherein the filter is configured to remove a type of sensitive data from the first log file;
exporting, from the first pod and in response to the filtering, the first log file to a log storage on the node, wherein the log storage is outside of the first pod wherein a second container in the second pod can access the log storage on the node; and
rotating, in response to an event, the first log file from the log storage, wherein the event is closing the first pod and the rotating includes removing the first log file from the node.
2 . The computer-implemented method of claim 1 , wherein the exporting includes storing the first log file in the log storage.
3 . The computer-implemented method of claim 1 , wherein the first log file stored in the log storage is accessible to a user account for the node.
4 . The computer-implemented method of claim 1 , further comprising:
generating a second log file, wherein the first log file is a standard output log and the second log file is a standard error log.
5 . The computer-implemented method of claim 4 , wherein the filter includes one or more providers, and a first provider is configured to remove the type of sensitive data, and a second provider is configured to remove a second type of sensitive data.
6 . The computer-implemented method of claim 5 , wherein the first log file and the second log file are filtered by a common set of providers.
7 . The computer-implemented method of claim 5 , wherein the first log file is filtered by the first provider and the second log file is filtered by the second provider.
8 . The computer-implemented method of claim 1 , wherein the filter includes a user interface configured to receive, from a data owner, one or more filter scripts, wherein each filter script is configured to remove a different type of sensitive data from the first log file.
9 . The computer-implemented method of claim 8 , wherein the each filter script can identify the different type of sensitive data and replace the different type of sensitive data with a generic value.
10 . A system comprising:
a processor;
a node of a cloud computing system configured to operate a plurality of pods received from one or more hosts, wherein the node is contained within a computing device, and the node includes a log storage; and
a computer-readable storage medium communicatively coupled to the processor and storing program instructions which, when executed by the processor, are configured to cause the processor to:
receive a first pod of the plurality of pods to the node of a cloud computing system, wherein the first pod:
includes a first container configured to run an application;
generates a first log file for the first container, wherein the first log file includes a set of actions performed the application for a period of time;
filters, by a filter, the first log file wherein the filter is configured to remove a type of sensitive data from the first log file; and
export, from the first pod and in response to the filtering of the first log file, the first log file to the log storage, wherein the log storage is outside of the first pod;
receive a second pod of the plurality of pods to the node of the cloud computing system, wherein a second container in the second pod can access the log storage on the node; and
rotate, in response to an event, the first log file from the log storage, wherein the event is closing the first pod and the rotating includes removing the first log file from the node.
11 . The system of claim 10 , wherein the node is configured to store a set of log files generated on the node, include the first log file in response to the export of the first log file.
12 . The system of claim 11 , wherein the set of log files is accessible to an account authorized to access the node.
13 . The system of claim 10 , wherein the filter includes a series of two or more filter scripts, and each filter script is configured to remove a different type of data from the first log file.
14 . A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processing unit to cause the processing unit to:
deploy a first pod on a node of a cloud computing system, wherein the first pod includes a first container configured to run an application and the node is contained within a computing device;
deploy a second pod on the node within the computing device;
generate a first log file for the first container, wherein the first log file includes a set of actions performed the application for a period of time;
filter, by a filter, the first log file wherein the filter is configured to remove a type of sensitive data from the first log file;
export, from the first pod and in response to the filtering, the first log file to a log storage on the node, wherein the log storage is outside of the first pod wherein a second container in the second pod can access the log storage on the node; and
rotate, in response to an event, the first log file from the log storage, wherein the event is closing the first pod and the rotating includes removing the first log file from the node.
15 . The computer program product of claim 14 , wherein the program instructions are further configured to cause the processing unit to:
generate a second log file, wherein the first log file is a standard output log and the second log file is a standard error log.
16 . The computer program product of claim 15 , wherein the filter includes one or more providers, and a first provider is configured to remove the type of sensitive data, and a second provider is configured to remove a second type of sensitive data.
17 . The computer program product of claim 16 , wherein the first log file and the second log file are filtered by a common set of providers.