Granted Patent B1

US 11,461,298 · App. 17/407,559 · Granted Oct 4, 2022

Inventors: Benjamin Shemmer (Brooklyn, NY); Nitay Joffe (New Rochelle, NY)

Assignee: ACTIONIQ, INC.

G06F16/215G06F16/2462G06F17/11

Abstract

Claims (63)

1. A computer-implemented method performed by a data processing apparatus, the method comprising:

receiving, by a computing device, a first data set, a statistical profile of a third data set, and identity resolution scores generated for the first data set;

generating, by the computing device, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals;

matching, by the computing device, the identity resolution scores generated for the first data set to pairs of records from the second data set;

generating, by the computing device, using regression, new scoring parameters from the second data set and the identity resolution scores matched to the pairs of records from the second data set; and

generating, by the computing device, identity resolution scores for pairs of records from the third data set using the new scoring parameters.

2. The method of claim 1 , wherein generating, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals, further comprises:

determining based on a comparison of a statistical profile of the first data set to a statistical profile of the second data set that the statistical profile of the first data set does not match the statistical profile of the second data set; and

in response to the determination that the statistical profile of the first data set does not match the statistical profile of the second data set, modifying the first data set by removing all occurrences of an example of valid data from the same columns of records of the first data set or removing a record with at least one column with invalid data from the first data set.

3. The method of claim 1 , wherein generating, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals, further comprises:

determining based on a comparison of a statistical profile of the first data set to a statistical profile of the second data set that the statistical profile of the first data set matches the statistical profile of the second data set; and

in response to the determination that the statistical profile of the first data set does matches the statistical profile of the second data set, storing a copy of the first data set as the second data set and storing a copy of the statistical profile of the first data set as the statistical profile of the second data set.

4. The method of claim 1 , wherein the first data set is a known data set for which the identity resolution scores have already been generated, the second data set is a mimic data set that mimic a target data set, and the third data set is the target data set.

5. The method of claim 1 , wherein matching identity resolution scores generated for the first data set to pairs of records from the second data set further comprises:

generating indicator data from the second data set, wherein the indicator data comprises records, and wherein each of the records of the indicator data comprises values indicating whether data from common columns in a pair of records from the second data set is the same or differs by less than a threshold value for the features of the common columns;

determining, for each one of the records in the indicator data for the second data set, from the identity resolution scores generated for the first data set, an identity resolution score that was generated for a pair of records from the first data set that correspond to the pair of records from the one of the records in the indicator data for the second data set; and

storing each of the identity resolutions scores determined for each one of the records in the indicator data for the second data set as the identity resolution scores for the second data set.

6. The method of claim 5 , wherein generating, using regression, new scoring parameters from the second data set and the identity resolution scores matched to the pairs of records from the second data set further comprises:

generating an equation based on a scoring equation for identity resolution, the indicator data for the second data set, and the identity resolutions scores for the second data set;

performing a log transformation on the equation to generate a log transformed equation; and

generating new scoring parameters by performing regression on the log transformed equations.

7. The method of claim 1 , further comprising combining one or more records of the third data set based on the identity resolution scores for pairs of records from the third data set.

8. The method of claim 1 , wherein the new scoring parameters comprise penalty factors for use with a scoring equation.

9. A computer-implemented system comprising:

a storage; and

a processor that receives a first data set, a statistical profile of a third data set, and identity resolution scores generated for the first data set;

generates, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals,

matches the identity resolution scores generated for the first data set to pairs of records from the second data set,

generates, using regression, new scoring parameters from the second data set and the identity resolution scores matched to the pairs of records from the second data set, and

generates identity resolution scores for pairs of records from the third data set using the new scoring parameters.

10. The system of claim 9 , wherein the processor generates, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals, by determining based on a comparison of a statistical profile of the first data set to a statistical profile of the second data set that the statistical profile of the first data set does not match the statistical profile of the second data set, and

in response to the determination that the statistical profile of the first data set does not match the statistical profile of the second data set, modifying the first data set by removing all occurrences of an example of valid data from the same columns of records of the first data set or removing a record with at least one column with invalid data from the first data set.

11. The system of claim 9 , wherein the processor generates, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals, by determining based on a comparison of a statistical profile of the first data set to a statistical profile of the second data set that the statistical profile of the first data set matches the statistical profile of the second data set, and

in response to the determination that the statistical profile of the first data set does matches the statistical profile of the second data set, storing a copy of the first data set as the second data set and storing a copy of the statistical profile of the first data set as the statistical profile of the second data set.

12. The system of claim 9 , wherein the first data set is a known data set for which the identity resolution scores have already been generated, the second data set is a mimic data set that mimic a target data set, and the third data set is the target data set.

13. The system of claim 9 , wherein the processor matches identity resolution scores generated for the first data set to pairs of records from the second data set by generating indicator data from the second data set, wherein the indicator data comprises records, and wherein each of the records of the indicator data comprises values indicating whether data from common columns in a pair of records from the second data set is the same or differs by less than a threshold value for the features of the common columns,

determining, for each one of the records in the indicator data for the second data set, from the identity resolution scores generated for the first data set, an identity resolution score that was generated for a pair of records from the first data set that correspond to the pair of records from the one of the records in the indicator data for the second data set, and

storing each of the identity resolutions scores determined for each one of the records in the indicator data for the second data set as the identity resolution scores for the second data set.

14. The system of claim 13 , wherein the processor generates, using regression, new scoring parameters from the second data set and the identity resolution scores matched to the pairs of records from the second data set by generating an equation based on a scoring equation for identity resolution, the indicator data for the second data set, and the identity resolutions scores for the second data set,

performing a log transformation on the equation to generate a log transformed equation, and

generating new scoring parameters by performing regression on the log transformed equations.

15. The system of claim 9 , further wherein the processor further combines one or more records of the third data set based on the identity resolution scores for pairs of records from the third data set.

16. The system of claim 9 , wherein the new scoring parameters comprise penalty factors for use with a scoring equation.

17. A system comprising: one or more computers and one or more storage devices storing instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving, by a computing device, a first data set, a statistical profile of a third data set, and identity resolution scores generated for the first data set;

generating, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals;

matching the identity resolution scores generated for the first data set to pairs of records from the second data set;

generating, using regression, new scoring parameters from the second data set and the identity resolution scores matched to the pairs of records from the second data set; and

generating identity resolution scores for pairs of records from the third data set using the new scoring parameters.

18. The system of claim 17 , wherein the instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising generating, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals, further comprise instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

determining based on a comparison of a statistical profile of the first data set to a statistical profile of the second data set that the statistical profile of the first data set does not match the statistical profile of the second data set; and

in response to the determination that the statistical profile of the first data set does not match the statistical profile of the second data set, modifying the first data set by removing all occurrences of an example of valid data from the same columns of records of the first data set or removing a record with at least one column with invalid data from the first data set.

19. The system of claim 17 , wherein the instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising generating, from the first data set, a second data set that has a statistical profile that matches the statistical profile of the third data set, each of the first data set, the second data set, and the third data set comprising records that comprise data about individuals, further comprise instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

determining based on a comparison of a statistical profile of the first data set to a statistical profile of the second data set that the statistical profile of the first data set matches the statistical profile of the second data set; and

in response to the determination that the statistical profile of the first data set does matches the statistical profile of the second data set, storing a copy of the first data set as the second data set and storing a copy of the statistical profile of the first data set as the statistical profile of the second data set.

20. The system of claim 17 , wherein the instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising matching identity resolution scores generated for the first data set to pairs of records from the second data set, further comprise instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

generating indicator data from the second data set, wherein the indicator data comprises records, and wherein each of the records of the indicator data comprises values indicating whether data from common columns in a pair of records from the second data set is the same or differs by less than a threshold value for the features of the common columns;

determining, for each one of the records in the indicator data for the second data set, from the identity resolution scores generated for the first data set, an identity resolution score that was generated for a pair of records from the first data set that correspond to the pair of records from the one of the records in the indicator data for the second data set; and

storing each of the identity resolutions scores determined for each one of the records in the indicator data for the second data set as the identity resolution scores for the second data set.

21. The system of claim 20 , wherein the instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising generating, using regression, new scoring parameters from the second data set and the identity resolution scores matched to the pairs of records from the second data set further comprise instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

generating an equation based on a scoring equation for identity resolution, the indicator data for the second data set, and the identity resolutions scores for the second data set;

performing a log transformation on the equation to generate a log transformed equation; and

generating new scoring parameters by performing regression on the log transformed equations.

Assignments (7)

Cited By (12)

US 12,195,338 US 12,214,420 US 12,248,545 US 12,261,023 US 12,284,172 US 12,299,093 US 12,301,558 US 12,311,447 US 12,314,443 US 12,347,095 US 12,406,829 US 12,470,552