System and method to purge data in an agnostic manner using grains
A system and method to purge data from databases using grains. The system includes a database and an agnostic data purge system. The database stores a current data table, and the agnostic data purge system includes a processor which generates an identification identifying the current data table, associates the identification with a purge statement to perform a data purge of the current data table, and generates a control table entry associating the purge statement with a grain for the data purge. The database performs the data purge of the current data table upon execution of the extracted purge statement and the grain with the data purge is independent of and agnostic of a specification of a data management platform, and the processor generates an updated data table as the current data table based on the purge data and non-purge data. The method implements the system.
1 . A computer-based system, comprising:
a database storing a current data table; and
a data purge system independent of a data management system, the data purge system including:
a hardware-based processor operatively connected to the data management system and to the database, wherein the hardware-based processor is configured to generate an identification identifying the current data table, to associate the identification with a purge statement configured to perform a data purge of the current data table, and to generate a control table entry associating the purge statement with a grain for the data purge;
a clock configured to determine whether a predetermined update period has elapsed; and
a memory configured to store instructions and configured to provide the instructions to the hardware-based processor,
wherein the database, responsive to receiving the control table entry, extracts the purge statement and the grain from the control table entry, and performs the data purge of the current data table upon execution of the extracted purge statement and the grain to generate purge data and non-purge data, wherein the performance of the data purge is independent of the data management system based on the execution of the extracted purge statement and the grain,
wherein the hardware-based processor generates an updated data table as the current data table based on the purge data and non-purge data,
wherein in a case that the predetermined update period has elapsed, the hardware-based processor generates a new identification identifying the current data table, associating the identification with a new purge statement configured to perform a new data purge of the current data table, and generating a new control table entry associating the new purge statement with a new grain for the new data purge,
wherein in the case that the predetermined update period has elapsed, the database, responsive to receiving the new control table entry, extracts the new purge statement and the new grain from the new control table entry, and performs the new data purge of the current data table upon execution of the extracted new purge statement and the new grain to generate new purge data and new non-purge data, wherein the performance of the new data purge is independent of the data management system based on execution of the extracted new purge statement and the new grain,
wherein in the case that the predetermined update period has elapsed, the hardware-based processor generates a new updated data table based on the new purge data and the new non-purge data, and
wherein in a case that the predetermined update period has not elapsed, the hardware-based processor checks the clock to determine whether the predetermined update period has elapsed.
2 . The computer-based system of claim 1 , wherein the hardware-based processor distributes the control table entry to the database.
3 . The computer-based system of claim 1 , wherein the clock is configured to associate a timestamp with the purge data and to determine whether a predetermined purge period has elapsed,
wherein in a case that the predetermined purge period has elapsed based on the timestamp, the hardware-based processor performs a final purge of the purge data, and
wherein in a case that the predetermined purge period has not yet elapsed, the clock determines whether a predetermined purge period has elapsed.
4 . The computer-based system of claim 3 , wherein the predetermined purge period is stored in a purge policy storage in the memory.
5 . The computer-based system of claim 1 , wherein the control table entry is stored in a control table in the memory.
6 . The computer-based system of claim 1 , wherein the grain is stored in a grain storage in the memory.
7 . The computer-based system of claim 1 , wherein the grain includes an identity indicator, a behavior indicator, and a state indicator.
8 . A computer-based method implemented by a data purge system, comprising:
(a) receiving, at a data management system, a current data table in a database, wherein the data purge system is independent of the data management system;
(b) generating an identification identifying the current data table;
(c) associating the identification with a purge statement configured to perform a data purge of the current data table;
(d) generating a control table entry associating the purge statement with a grain for the data purge;
(e) extracting, at the database, the purge statement and the grain from the control table entry;
(f) performing the data purge of the current data table in the database upon execution of the extracted purge statement and the grain to generate purge data and non-purge data, wherein the step (f) of performing the data purge is independent of the data management system based on execution of the extracted purge statement and the grain;
(g) generating an updated data table in the database based on the purge data and non-purge data;
(h) determining whether a predetermined update period has elapsed;
(i) in a case that the predetermined update period has elapsed in step (h), repeating steps (a)-(h); and
(j) in a case that the predetermined update period has not elapsed in step (h), repeating step (h).
9 . The computer-based method of claim 8 , further comprising, between steps (d) and (e):
(d1) distributing the control table entry to the database.
10 . The computer-based method of claim 8 , further comprising, between steps (f) and (g):
(f1) associating a timestamp with the purge data;
(f2) determining whether a predetermined purge period has elapsed based on the timestamp;
(f3) in a case that the predetermined purge period has elapsed, performing a final purge of the purge data; and
(f4) in a case that the predetermined purge period has not yet elapsed, loop back to step (f2).
11 . The computer-based method of claim 10 , further comprising, between steps (f1) and (f2):
(ff1) storing the predetermined purge period in a purge policy storage.
12 . The computer-based method of claim 8 , further comprising, between steps (d) and (e):
(d1) storing the control table entry in a control table.
13 . A non-transitory computer-readable storage medium storing instructions executable by a processor of a data purge system, the instructions comprising:
(a) receiving, at a data management system, a current data table in a database, wherein the data purge system is independent of the data management system;
(b) generating an identification identifying the current data table;
(c) associating the identification with a purge statement configured to perform a data purge of the current data table;
(d) generating a control table entry associating the purge statement with a grain for the data purge;
(e) extracting, at the database, the purge statement and the grain from the control table entry;
(f) performing the data purge of the current data table in the database upon execution the extracted purge statement and the grain to generate purge data and non-purge data, wherein the step (f) of performing the data purge is independent of the data management system based on execution of the extracted purge statement and the grain;
(g) generating an updated data table in the database based on the purge data and non-purge data;
(h) determining whether a predetermined update period has elapsed;
(i) in a case that the predetermined update period has elapsed in step (h), repeating steps (a)-(h); and
(i) in a case that the predetermined update period has not elapsed in step (h), repeating step (h).
14 . The non-transitory computer-readable storage medium of claim 13 , further comprising, between steps (d) and (e):
(d1) distributing the control table entry to the database.
15 . The non-transitory computer-readable storage medium of claim 13 , further comprising, between steps (f) and (g):
(f1) associating a timestamp with the purge data;
(f2) determining whether a predetermined purge period has elapsed based on the timestamp;
(f3) in a case that the predetermined purge period has elapsed, performing a final purge of the purge data; and
(f4) in a case that the predetermined purge period has not yet elapsed, loop back to step (f2).
16 . The non-transitory computer-readable storage medium of claim 15 , further comprising, between steps (f1) and (f2):
(ff1) storing the predetermined purge period in a purge policy storage.
17 . The non-transitory computer-readable storage medium of claim 13 , further comprising, between steps (d) and (e):
(d1) storing the control table entry in a control table.