System and method for shifting dates in the de-identification of datasets

1. A method of dataset de-identification, the method comprising:

retrieving, by a server, a dataset having a plurality of entries containing personal identifying information;

identifying by the server, date quasi-identifiers in the dataset for each of the plurality of entries;

performing, by the server, consolidation of a plurality of date events in the date quasi-identifiers and connected dates in the dataset;

identifying, by the server, an anchor date in a longitudinal sequence of the plurality of date events, wherein the anchor date is a front anchor date at a start of the longitudinal sequence to create a running difference between the plurality of date events in the longitudinal sequence to ensure an order of dates is preserved;

selecting, by the server, a probability distribution based upon a drift threshold, wherein the drift threshold is an anchor generalization interval minus a generalized difference interval and, wherein the probability distribution is selected to provide a date drift less than the threshold while achieving a desired risk threshold;

performing, by the server, randomization of the running difference with a defined interval for each defined running difference within the longitudinal sequence, wherein randomization is performed by the probability distribution;

performing, by the server, de-identification of the plurality of entries in the dataset including the date quasi-identifiers;

performing, by the server, risk analysis of the de-identified dataset to determine a risk metric;

iteratively performing, by the server, de-identification of the date quasi-identifiers until a defined risk threshold is met relative to the determined risk metric; and

storing the de-identification dataset in a database.

2. The method of 1 , wherein the anchor is a first date in the sequence of the plurality of date events.

3. The method of claim 1 , wherein the anchor date is a fixed date added to the longitudinal sequence of the plurality of date events.

4. The method of claim 1 further comprising generating a rear anchor date is a fixed date added to an end of the longitudinal sequence of the plurality of date events.

5. The method of claim 1 further comprising randomizing the running difference within a level of generalization need for de-identification of the dataset.

6. The method of claim 5 wherein the level of generalization is defined by a hierarchy based upon the risk analysis.

7. The method of claim 6 wherein the level of generalization is defined by an interval of days, the level of generalization defining an upper and lower bound for randomization based upon a selected level of de-identification.

8. The method of claim 1 , wherein randomization is performed by a uniform probability distribution, linear probability distribution or an exponential probability distribution.

9. The method of claim 1 , wherein randomization is performed by a histogram matching the running difference.

10. The method of claim 1 , further comprising summing a de-identified running difference to a de-identified anchor date to generate a sequence of de-identified dates therefore preserving a longitudinal order of date events.

11. The method of claim 10 further comprising identifying one or more date events as connected dates to an associated quasi-identifier date wherein the connected dates are modified based upon a de-identified running difference.

12. The method of claim 11 wherein the connected dates are modified by shifting them by a same number of days as de-identified date events they are associated with.

13. The method of claim 12 further comprising generating a consolidated date table comprising quasi-identifier date events from a plurality of data tables to generate a respective single quasi-identifier date to represent the plurality of date events.

14. The method of claim 13 further comprising adding connected dates to a consolidated date table with the quasi-identifier date they are associated with.

15. The method of claim 13 further comprising generating associating each quasi-identifier date event and connected dates with keys to originating date events wherein the keys identify a row within a source table of the dataset.

16. The method of claim 15 further comprising generating a core identifier for quasi-identifier date events, the core identifier defining a column where the quasi-identifier date resides with a source table.

17. The method of claim 16 further comprising performing de-identification using consolidated quasi-identifier date events by generating a destination table corresponding to a respective source table in the dataset using generated keys and core identifiers to map randomized date events to maintain the respective longitudinal sequences.

18. The method of claim 1 wherein risk analysis is determined based upon a level of k-anonymity determined for the dataset.

19. The method of claim 18 wherein performing de-identification of the dataset is performed by an optimal lattice method.

20. A system for de-identifying a dataset, the system comprising:

a memory storing a database, the memory further storing sets of program instructions;

a server coupled to the memory, wherein the server, upon executing the sets of program instructions, is configured to:

retrieve a dataset having a plurality of entries containing personal identifying information;

identify date quasi-identifiers in the dataset for each of the plurality of entries;

perform consolidation of a plurality of date events in the date quasi-identifiers and connected dates in the dataset;

identify an anchor date in a longitudinal sequence of the plurality of date events, wherein the anchor date is a front anchor date at a start of the longitudinal sequence to create a running difference between the plurality of date events in the longitudinal sequence to ensure an order of dates is preserved;

select a probability distribution based upon a drift threshold, where the drift threshold is an anchor generalization interval minus a generalized difference interval, and wherein the distribution is selected to provide a date drift less than the threshold while achieving a desired risk threshold;

perform randomization of the running difference with a defined interval for each defined running difference within the longitudinal sequence; wherein randomization is performed by the probability distribution;

perform de-identification of the plurality of entries in the dataset including the date quasi-identifiers;

perform risk analysis of the de-identified dataset to determine a risk metric;

perform de-identification of the date quasi-identifiers iteratively, until a defined risk threshold is met relative to the determined risk metric; and

store the de-identification dataset in the database.