Transaction-level data retention policy inheritance
A computer system provides transaction-level data retention policy inheritance. The system may perform operations including storing a first dataset comprising a plurality of transactions, each of the plurality of transactions comprising one or more data items; receiving a first transaction to the first dataset, the first transaction comprising one or more data items; determining a first retention policy for the first transaction; and storing the first retention policy with the first transaction. The system may further perform operations including calculating a deletion date for the first transaction based on the first retention policy; and storing the deletion date with the first transaction in the first dataset.
1 . A computer-implemented method for updating a retention policy for a transaction of a dataset, the computer-implemented method comprising, by one or more hardware processors executing program instructions:
receiving a request to update a first retention policy associated with a first transaction of a plurality of transactions, each of the plurality of transactions comprising one or more data items;
updating the first retention policy based on the request;
storing the updated first retention policy in association with the first transaction;
identifying, in a data source or service, information indicative of any downstream transactions or datasets associated with the first transaction, wherein the information includes an indication of a second transaction;
accessing a second retention policy associated with the second transaction;
updating the second retention policy based at least in part on the updated first retention policy; and
storing the updated second retention policy in association with the second transaction.
2 . The computer-implemented method of claim 1 , wherein identifying, in the data source, information indicative of any downstream transactions or datasets further comprises, by the one or more hardware processors executing program instructions:
querying the data source or service using a unique identifier associated with the first transaction, causing the data source or service to return the information indicative of any downstream transactions or datasets.
3 . The computer-implemented method of claim 1 further comprising, by the one or more hardware processors executing program instructions:
determining the first transaction is upstream from the second transaction.
4 . The computer-implemented method of claim 1 further comprising, by the one or more hardware processors executing program instructions:
accessing a deletion date associated with the first transaction;
calculating an updated deletion date based on the updated first retention policy; and
storing the updated deletion date in association with the first transaction.
5 . The computer-implemented method of claim 1 further comprising, by the one or more hardware processors executing program instructions:
accessing a second deletion date associated with the second transaction;
calculating a second updated deletion date based on the updated second retention policy; and
storing the second updated deletion date in association with the second transaction.
6 . The computer-implemented method of claim 5 further comprising, by the one or more hardware processors executing program instructions:
deleting the second transaction based on the second updated deletion date and/or the updated second retention policy.
7 . The computer-implemented method of claim 5 , wherein a first dataset includes a plurality of transactions, wherein the plurality of transactions includes the first transaction and the second transaction, and wherein each transaction of the plurality of transactions is associated with a retention policy and a calculated deletion date.
8 . The computer-implemented method of claim 7 further comprising, by the one or more hardware processors executing program instructions:
generating data useable for rendering a graphical user interface, the graphical user interface including at least one of:
indications of deletion dates and/or retention policies associated with a plurality of datasets or transactions, or
user input elements for creating or selecting a retention policy to apply to a dataset or transaction.
9 . The computer-implemented method of claim 1 , wherein the first transaction is in a first dataset, wherein the second transaction is in a second dataset, and wherein the first dataset is different from the second dataset.
10 . The computer-implemented method of claim 1 further comprising, by the one or more hardware processors executing program instructions:
calculating a deletion date for the second transaction based on a minimum deletion date associated with the updated second retention policy and the updated first retention policy; and
storing the deletion date in association with the second transaction.
11 . The computer-implemented method of claim 1 , wherein:
the updated first retention policy is at least one of: a time to live policy, or a fixed deletion date policy, and
the updated second retention policy a different one of at least one of: a time to live policy, or a fixed deletion date policy.
12 . The computer-implemented method of claim 1 further comprising, by the one or more hardware processors executing program instructions:
in response to updating the second retention policy:
querying a second data source or service using a unique identifier associated with the second transaction; and
receiving, from the second data source or service, information indicative of any downstream transactions or datasets associated with the second transaction, wherein the information includes an indication of a third transaction, wherein the third transaction was not previously identified as a downstream transaction in the data source or service.
13 . The computer-implemented method of claim 12 further comprising, by the one or more hardware processors executing program instructions:
accessing a third retention policy associated with the third transaction in the second data source or service;
updating the third retention policy based on the updated first retention policy; and
storing the updated third retention policy in association with the third transaction.
14 . The computer-implemented method of claim 1 , wherein the information indicative of any downstream transactions or datasets associated with the first transaction includes an identifier for a first dataset, and wherein the computer-implemented method further comprises, by the one or more hardware processors executing program instructions:
accessing a plurality of retention policies of the first dataset;
updating the plurality of retention policies based at least in part on the updated first retention policy; and
storing each of the updated plurality of retention policies in association with an associated transaction of a plurality of transactions included in the first dataset.
15 . The computer-implemented method of claim 1 further comprising, by the one or more hardware processors executing program instructions:
identifying, in the data source or service, information indicative of any upstream transactions or datasets associated with the first transaction, wherein the information includes an indication of a third transaction; and
accessing a third retention policy associated with the third transaction,
wherein updating the second retention policy is further based at least in part on the third retention policy.
16 . The computer-implemented method of claim 1 further comprising, by the one or more hardware processors executing program instructions:
determining a compatibility between the updated first retention policy and the second retention policy; and
in response to determining the compatibility, severing a connection between the first transaction and the second transaction.
17 . The computer-implemented method of claim 1 , wherein the request to update the first retention policy includes an override policy, and wherein updating the second retention policy includes severing a connection between the first transaction and the second transaction.
18 . The computer-implemented method of claim 1 , wherein the updated second retention policy indicates the second transaction is to be deleted, and wherein the computer-implemented method further comprises, by the one or more hardware processors executing program instructions:
deleting the second transaction.
19 . A system comprising:
one or more computer-readable storage mediums storing program instructions; and
one or more processors configured to execute the program instructions to cause the system to:
receive a request to update a first retention policy associated with a first transaction of a plurality of transactions, each of the plurality of transactions comprising one or more data items;
update the first retention policy based on the request;
storing the updated first retention policy in association with the first transaction;
identify, in a data source or service, information indicative of any downstream transactions or datasets associated with the first transaction, wherein the information includes an indication of a second transaction;
access a second retention policy associated with the second transaction;
update the second retention policy based at least in part on the updated first retention policy; and
store the updated second retention policy in association with the second transaction.
20 . A computer program product comprising one or more computer-readable storage mediums storing program instructions, the program instructions executable by one or more processors to:
receive a request to update a first retention policy associated with a first transaction of a plurality of transactions, each of the plurality of transactions comprising one or more data items;
update the first retention policy based on the request;
storing the updated first retention policy in association with the first transaction;
identify, in a data source or service, information indicative of any downstream transactions or datasets associated with the first transaction, wherein the information includes an indication of a second transaction;
access a second retention policy associated with the second transaction;
update the second retention policy based at least in part on the updated first retention policy; and
store the updated second retention policy in association with the second transaction.