Smart de-identification using date jittering
System and method to produce an anonymized cohort having less than a predetermined risk of re-identification. The method includes receiving a data query of requested traits for the anonymized cohort, querying a data source to find records that possess at least some of the traits, forming a dataset from at least some of the records, and grouping the dataset in time into a first boundary group, a second boundary group, and one or more non-boundary groups temporally between the first boundary group and second boundary group. For each non-boundary group, calculating maximum time limits the non-boundary group can be time-shifted without overlapping an adjacent group, calculating a group jitter amount, capping the group jitter amount by the maximum time limits and by respective predetermined jitter limits, and jittering said non-boundary group by the capped group jitter amount to produce an anonymized dataset. Return the anonymized dataset.
1. A method of obscuring identity of members in a dataset, the method comprising steps of:
grouping, by a processor into a plurality of groups, dates associated with the members and that are within a preset time interval of each other; and
measuring, by the processor, forward and backward distance gaps between the groups on a date scale;
calculating, by the processor, forward and backward jitter limits of the forward and backward distance gaps adjacent to each of the plurality of groups and respective forward and backward jitter shift parameters;
calculating, by the processor, a group jitter amount, using an earliest date in each of the plurality of groups; and
jittering, by the processor, only these of the plurality of groups that have both the forward and backward distance gaps by the group jitter.
2. The method of claim 1 , further comprising capping the group jitter by respective predetermined configurable forward and backward jitter limits.
3. The method of claim 1 , further comprising a step of date-shifting, by the processor, the dates in each of the plurality of groups, wherein date-shifting of the plurality of groups is performed together to maximize an associated analytic value.
4. The method of claim 2 , wherein the predetermined forward and backward jitter limits are dependent upon one or more characteristics of the data and are dependent intended uses of the data.
5. The method of claim 2 , wherein the group jitter amount is within a range established by the forward and backward jitter limits.
6. The method of claim 2 , wherein the predetermined forward and backward jitter limits are different for a first non-boundary group and a second non-boundary group.
7. The method of claim 2 , wherein the group jitter amount is calculated using a hash function.
8. The method of claim 7 , wherein the hash function output is scaled to the capped group jitter.
9. The method of claim 7 , wherein the hash function hashes a date value from each respective non-boundary group by use of a secret key.
10. The method of claim 9 , wherein the secret key comprises a project sub-key specific to a project, and an entity sub-key specific to an entity whose data is being processed.
11. The method of claim 10 , wherein the project sub-key comprises a randomly-generated universally unique identifier (UUID).
12. The method of claim 9 , wherein the minimum jitter limit allows the non-boundary group to be time-shifted without overlapping an adjacent group.
13. The method of claim 1 , wherein the group jitter amount is calculated using a deterministic function.
14. The method of claim 1 , further comprising a step of calculating forward and backward minimum and maximum jitter limits as a function of a desired analytic value.
15. The method of claim 1 , wherein the jittering is not performed on the boundary groups to guarantee incremental ordering, because only one adjacent gap can be calculated.
16. A system of obscuring identity of members in a dataset, the system comprising:
a processor configured to:
group into a plurality of groups, dates associated with the members and that are within a preset time interval of each other;
measure, forward and backward distance gaps between the groups on a date scale;
calculate forward and backward jitter limits of the forward and backward distance gaps adjacent to each of the plurality of groups and respective forward and backward jitter shift parameters;
calculate a group jitter amount, using an earliest date in each of the plurality of groups;
jitter, using the group jitter, only these of the plurality of groups that have both the forward and backward distance gaps.
17. The system of claim 16 , wherein the group jitter amount is calculated using a hash function.
18. The system of claim 17 , wherein the hash function hashes a date value from each respective non-boundary group by use of a secret key, wherein the secret key comprises a project sub-key specific to a project, and an entity sub-key specific to an entity whose data is being processed.
19. The system of claim 16 , wherein the processor is further configured to date-shift the dates in each of the plurality of groups together to maximize an associated analytic value.
20. A method comprising:
partitioning, by a processor, data into time-separated groups;
shifting, by the processor, the time-separated groups by a fixed amount of time;
determining, by the processor, a forward and backward jitter limit for each of the time separated groups shifted by the fixed amount of time; and
calculating, by the processor, a jitter amount for each of the time-separated groups.