Dynamic database redaction using protected secret material
Techniques for dynamic database redaction using protected encryption secret material are described. A masking policy is defined that includes a reference to a secret material stored by a secrets manager service. The masking policy further identifies a pseudonymous redaction function that utilizes a cryptographic function requiring such a secret material. The secrets manager service is configured to grant access to the secret material by an entity of the database service that executes queries, such as a leader node of a cluster. For a particular query, the cluster obtains the secret material from the secrets manager service in a secure manner, uses the secret material for applying the cryptographic function to values for redaction purposes, and deletes any copies of secret material thereafter.
1. A computer-implemented method comprising:
receiving, at a database service, a request to create a masking policy, the request identifying a pseudonymous redaction function implemented by the database service, the request further including an identifier of a secret material that is stored in a separate system that is separate from the database service;
receiving, at the database service, a request to attach the masking policy to a column of a table of a database;
receiving, at the database service, a request to execute a query involving at least a column of a relation of the database; and
executing the query to generate a result, comprising modifying values of the column based on use of the secret material and the pseudonymous redaction function of the masking policy to yield modified values for the result, wherein executing the query comprises:
obtaining a copy of the secret material from the separate system, by a leader node of a cluster of the database service, via use of the identifier of the secret material;
transmitting the copy of the secret material to each of one or more compute nodes of the cluster, via an encrypted channel or message, for use in modifying values of the column;
deleting the copy of the secret material by the leader node; and
deleting the copies of the secret material by the one or more compute nodes.
2. The computer-implemented method of claim 1 , wherein:
the separate system comprises a secrets manager service implemented within a multi-tenant service provider network; and
a user stored the secret material with the secrets manager service and configured a permission for the leader node to access the secret material.
3. The computer-implemented method of claim 1 , wherein:
the pseudonymous redaction function includes use of a hash-based message authentication code (HMAC) function;
the secret material itself is not within the masking policy; and
the secret material is not directly accessible by a user that issued the query.
4. A computer-implemented method comprising:
receiving, at a database service, a request to execute a query involving at least a column of a relation of a database; and
executing the query to generate a result, comprising modifying values of the column based on use of secret material and a pseudonymous redaction function implemented by the database service to yield modified values for the result, wherein executing the query comprises obtaining a copy of the secret material, from a separate system that is separate from the database service, via use of an identifier of the secret material.
5. The computer-implemented method of claim 4 , wherein the separate system comprises a secrets manager service implemented within a multi-tenant service provider network.
6. The computer-implemented method of claim 5 , further comprising:
receiving, at the secrets manager service, a request to store the secret material or to generate and store the secret material;
encrypting, by the secrets manager service, the secret material; and
storing the encrypted secret material in a non-volatile storage medium.
7. The computer-implemented method of claim 6 , further comprising configuring, by the secrets manager service, an entity of the database service to have permission to access the secret material, wherein the entity is one of a leader node of a cluster of the database service, the cluster of the database service, or a compute node of the database service.
8. The computer-implemented method of claim 7 , wherein obtaining the copy of the secret material comprises:
determining, based on a context of the query, an identity or credential to use to obtain the secret material, and
wherein the obtaining the copy of the secret material includes sending a request, by the leader node of the cluster to the secrets manager service, for the secret material via use of the determined identity or credential.
9. The computer-implemented method of claim 8 , wherein executing the query comprises transmitting the secret material by the leader node to one or more compute nodes of the cluster via one or more encrypted channels or messages.
10. The computer-implemented method of claim 9 , further comprising deleting the secret material, by the leader node, after the transmitting of the secret material by the leader node.
11. The computer-implemented method of claim 10 , wherein each of the one or more compute nodes utilizes the secret material to execute the pseudonymous redaction function and thereafter deletes the secret material.
12. The computer-implemented method of claim 7 , further comprising rotating the secret material, by the secrets manager service, to instead designate an updated secret material for use by the entity of the database service.
13. The computer-implemented method of claim 4 , wherein the modifying of the values of the column occurs based on a selected masking policy, wherein the secret material itself is not within the masking policy, and wherein the secret material is not directly accessible by a user that issued the query.
14. The computer-implemented method of claim 4 , further comprising:
receiving a request to create a masking policy, the request identifying the pseudonymous redaction function implemented by the database service, the request further including an identifier of the secret material that is stored in the separate system;
receive a request to attach the masking policy to the relation of a database; and
attaching the masking policy to the relation, comprising updating one or more data structures to associate the masking policy with the relation.
15. A system comprising:
a first one or more electronic devices to implement a secrets manager service in a multi-tenant service provider network; and
a second one or more electronic devices to implement a database service in the multi-tenant service provider network, the database service including instructions that upon execution cause the database service to:
receive a request to execute a query involving at least a column of a relation of a database; and
execute the query to generate a result, comprising modifying values of the column based on use of a secret material and a pseudonymous redaction function to yield modified values for the result, wherein to execute the query the database service is at least to obtain a copy of the secret material, from the secrets manager service that is separate from the database service, via use of an identifier of the secret material.
16. The system of claim 15 , wherein the secret manager service is to:
receive a request to store the secret material or to generate and store the secret material;
encrypt the secret material; and
store the encrypted secret material in a non-volatile storage medium.
17. The system of claim 16 , wherein the secret manager service is further to configure an entity of the database service to have permission to access the secret material, wherein the entity is one of a leader node of a cluster of the database service, the cluster of the database service, or a compute node of the database service.
18. The system of claim 17 , wherein the database service, to obtain the copy of the secret material, is to:
send a request, by the leader node of the cluster to the secrets manager service, for the secret material, wherein the request is sent with an identity of the leader node that allows the secrets manager service to authorize the leader node as having permission to access the secret material; and
receive, by the leader node from the secrets manager service, the secret material via an encrypted channel or message.
19. The system of claim 18 , wherein the database service, to execute the query, is to transmit the secret material by the leader node to one or more compute nodes of the cluster via one or more encrypted channels or messages.
20. The system of claim 15 , wherein the database service further includes instructions that when executed cause the database service to:
receive a request to create a masking policy, the request identifying the pseudonymous redaction function implemented by the database service, the request further including an identifier of the secret material that is stored in the secret manager service but not including the secret material itself;
receive a request to attach the masking policy to the relation of a database; and
attach the masking policy to the relation, comprising updating one or more data structures to associate the masking policy with the relation.