Automated data compliance and observability
In one embodiment, an observability and assurance service, associated with various clusters of application services for an application that are executed in a data mesh, may configure a data compliance filter for a particular application service in one of the clusters of application services according to a data compliance policy. The observability and assurance service may monitor the data and traffic associated with the particular application service, wherein the data compliance filter is applied to the traffic to restrict sensitive data in the traffic from being processed by the particular application service. The observability and assurance service may make a determination that the data compliance policy has been violated by the particular application service. The observability and assurance service may modify, based on the determination, the data compliance filter for the particular application service.
1. A method comprising:
configuring, by an observability and assurance service associated with various clusters of application services for an application that are executed in a data mesh, a data compliance filter for a particular application service in one of the various clusters of application services according to a data compliance policy that indicates a targeted compliance state for the various clusters of application services according by region;
monitoring, by the observability and assurance service, data and traffic associated with the particular application service, wherein the data compliance filter is applied to the traffic to restrict sensitive data in the traffic from being processed by the particular application service;
making, by the observability and assurance service, a determination that the data compliance policy has been violated by the particular application service; and
modifying, by the observability and assurance service and based on the determination, the data compliance filter for the particular application service.
2. The method as in claim 1 , wherein the traffic is from another application service within the various clusters of application services.
3. The method as in claim 1 , wherein the traffic is sent from outside the various clusters of application services.
4. The method as in claim 1 , wherein the data compliance filter comprises one or more data compliance rules that restrict a portion of the sensitive data from being processed by the particular application service based on a geographic location associated with the sensitive data and a geographic location associated with the particular application service.
5. The method as in claim 1 , further comprising:
reporting, by the observability and assurance service, the determination that the data compliance policy has been violated.
6. The method as in claim 1 , wherein the data compliance filter is based in part on an annotation in program code for the application that indicates a category of sensitive data.
7. The method as in claim 1 , wherein the determination indicates that a specific type of sensitive data in the traffic is being processed by the particular application service.
8. The method as in claim 7 , wherein modifying the data compliance filter causes the specific type of sensitive data to be prevented from being processed by the particular application service.
9. The method as in claim 1 , wherein the data compliance filter is a data traffic filter that:
detects the traffic associated with the particular application service arriving at a port associated with the particular application service; and
makes a routing decision for the traffic based on one or more data compliance rules.
10. An apparatus, comprising:
one or more network interfaces;
a processor coupled to the one or more network interfaces and configured to execute one or more processes; and
a memory configured to store a process that is executable by the processor, the process when executed configured to:
configure, by an observability and assurance service associated with various clusters of application services for an application that are executed in a data mesh, a data compliance filter for a particular application service in one of the various clusters of application services according to a data compliance policy that indicates a targeted compliance state for the various clusters of application services according by region;
monitor data and traffic associated with the particular application service, wherein the data compliance filter is applied to the traffic to restrict sensitive data in the traffic from being processed by the particular application service;
make a determination that the data compliance policy has been violated by the particular application service; and
modify, based on the determination, the data compliance filter for the particular application service.
11. The apparatus as in claim 10 , wherein the traffic is from another application service within the various clusters of application services.
12. The apparatus as in claim 10 , wherein the traffic is sent from outside the various clusters of application services.
13. The apparatus as in claim 10 , wherein the data compliance filter comprises one or more data compliance rules that restrict a portion of the sensitive data from being processed by the particular application service based on a geographic location associated with the sensitive data and a geographic location associated with the particular application service.
14. The apparatus as in claim 10 , wherein the process when executed is further configured to:
report, by the observability and assurance service, the determination that the data compliance policy has been violated.
15. The apparatus as in claim 10 , wherein the data compliance filter is based in part on an annotation in program code for the application that indicates a category of sensitive data.
16. The apparatus as in claim 10 , wherein the determination indicated that a specific type of sensitive data in the traffic is being processed by the particular application service.
17. The apparatus as in claim 16 , wherein the process when executed is configured to modify the data compliance filter by causing the specific type of sensitive data to be prevented from being processed by the particular application service.
18. A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:
configuring, by an observability and assurance service associated with various clusters of application services for an application that are executed in a data mesh, a data compliance filter for a particular application service in one of the various clusters of application services according to a data compliance policy that indicates a targeted compliance state for the various clusters of application services according by region;
monitoring, by the observability and assurance service, data and traffic associated with the particular application service, wherein the data compliance filter is applied to the traffic to restrict sensitive data in the traffic from being processed by the particular application service;
making, by the observability and assurance service, a determination that the data compliance policy has been violated by the particular application service; and
modifying, by the observability and assurance service and based on the determination, the data compliance filter for the particular application service.