Entity resolution techniques and systems

1. A method, comprising:

performing, by at least one data processing device:

(a) obtaining a plurality of data sets representing a respective plurality of entities;

(b) selecting, from the data sets, a first data set representing a first entity, wherein the first data set includes a plurality of values of a respective plurality of descriptors of the first entity;

(c) selecting two or more of the descriptors of the first entity, wherein the first data set includes respective values of the selected descriptors of the first entity;

(d) for each of the selected descriptors, determining an individual probability that the value of the descriptor describes an entity in a particular population of entities;

(e) estimating a joint probability that the values of the two or more selected descriptors of the first entity describe a second entity in the particular population of entities, different from the first entity, wherein the joint probability is estimated based, at least in part, on the individual probabilities;

(f) determining whether the joint probability of the values of the selected descriptors is less than a threshold probability;

(g) if the joint probability of the values of the selected descriptors is less than the threshold probability, searching in the plurality of data sets for a second data set including the values of the selected descriptors;

(h) if the second data set is found, performing an entity resolution operation, wherein performing the entity resolution operation comprises determining, based at least in part on the joint probability of the values of the selected descriptors being less than the threshold probability and on the first and second data sets including the values of the selected descriptors, that the second data set describes the first entity; and

repeating steps (b)-(h) one or more times,

wherein a false positive rate of the entity resolution operation is determined based on a size of the population and on the threshold probability, and

wherein estimating the joint probability that the values of all the selected descriptors describe a second entity in the particular population of entities comprises multiplying the individual probabilities of the values of the selected descriptors.

2. The method of claim 1 , wherein determining the individual probability that the value of the descriptor describes an entity in the particular population of entities comprises counting occurrences of the descriptor value in a plurality of data sets representing entities in the population or in a sample of the population.

3. The method of claim 1 , wherein the plurality of descriptors includes at least one descriptor selected from the group of descriptor types consisting of personal descriptors, educational descriptors, and professional descriptors.

4. The method of claim 1 , wherein the plurality of descriptors includes at least one descriptor of a profile of the first entity.

5. The method of claim 4 , wherein the profile of the first entity is a social media profile of the first entity.

6. The method of claim 1 , wherein selecting the two or more descriptors comprises using machine learning to select the two or more descriptors.

7. The method of claim 1 , wherein the selected descriptors include first and second descriptors, wherein the values of the first and second descriptors are not independent, and wherein estimating the joint probability further comprises multiplying a product of the individual probabilities of the values of the first and second descriptors by a corrective factor.

8. The method of claim 1 , further comprising:

calculating the false positive rate of the entity resolution operation based on the size of the population and on the threshold probability; and

determining whether the false positive rate of the entity resolution operation is lower or higher than a target false positive entity resolution rate, and if so, changing the threshold probability.

9. The method of claim 1 , further comprising setting the threshold probability based, at least in part, on the size of the population.

10. The method of claim 9 , wherein setting the threshold probability based, at least in part, on the size of the population comprises setting the threshold probability to a value within one order of magnitude of the inverse of the size of the population.

11. The method of claim 1 , wherein the second data set is derived from a resume and/or a social media profile.

12. The method of claim 1 , wherein performing the entity resolution operation further comprises combining the first and second data sets, at least in part.

13. The method of claim 12 , further comprising: identifying one or more entities included in the plurality of entities as candidates for a job by searching the plurality of data sets for one or more respective data sets matching search criteria.

14. The method of claim 13 , wherein the search criteria include one or more qualifications for the job and one or more preferences of a recruiter.

15. The method of claim 13 , further comprising: ranking the candidates and/or assigning the candidates to tiers.

16. A system, comprising:

at least one memory device storing computer-readable instructions; and

at least one data processing device operable to execute the computer-readable instructions to perform operations including:

(a) obtaining a plurality of data sets representing a respective plurality of entities;

(b) selecting, from the data sets, a first data set representing a first entity, wherein the first data set includes a plurality of values of a respective plurality of descriptors of the first entity;

(c) selecting two or more of the descriptors of the first entity, wherein the first data set includes respective values of the selected descriptors of the first entity;

(d) for each of the selected descriptors, determining an individual probability that the value of the descriptor describes an entity in a particular population of entities;

(e) estimating a joint probability that the values of the two or more selected descriptors of the first entity describe a second entity in the particular population of entities, different from the first entity, wherein the joint probability is estimated based, at least in part, on the individual probabilities;

(f) determining whether the joint probability of the values of the selected descriptors is less than a threshold probability;

(g) if the joint probability of the values of the selected descriptors is less than the threshold probability, searching in the plurality of data sets for a second data set including the values of the selected descriptors;

(h) if the second data set is found, performing an entity resolution operation, wherein performing the entity resolution operation comprises determining, based at least in part on the joint probability of the values of the selected descriptors being less than the threshold probability and on the first and second data sets including the values of the selected descriptors, that the second data set describes the first entity; and

repeating steps (b)-(h) one or more times,

wherein a false positive rate of the entity resolution operation is determined based on a size of the population and on the threshold probability, and

wherein estimating the joint probability that the values of all the selected descriptors describe a second entity in the particular population of entities comprises multiplying the individual probabilities of the values of the selected descriptors.

17. The system of claim 16 , wherein determining the individual probability that the value of the descriptor describes an entity in the particular population of entities comprises counting occurrences of the descriptor value in a plurality of data sets representing entities in the population or in a sample of the population.

18. The system of claim 16 , wherein the selected descriptors include first and second descriptors, wherein the values of the first and second descriptors are not independent, and wherein estimating the joint probability further comprises multiplying a product of the individual probabilities of the values of the first and second descriptors by a corrective factor.

19. The system of claim 16 , wherein the operations further include setting the threshold probability based, at least in part, on the size of the population.

20. The system of claim 19 , wherein setting the threshold probability based, at least in part, on the size of the population comprises setting the threshold probability to a value within one order of magnitude of the inverse of the size of the population.

21. A computer-readable storage medium having instructions stored thereon that, when executed by a data processing device, cause the data processing device to perform operations comprising:

(a) obtaining a plurality of data sets representing a respective plurality of entities;

(b) selecting, from the data sets, a first data set representing a first entity, wherein the first data set includes a plurality of values of a respective plurality of descriptors of the first entity;

(c) selecting two or more of the descriptors of the first entity, wherein the first data set includes respective values of the selected descriptors of the first entity;

(d) for each of the selected descriptors, determining an individual probability that the value of the descriptor describes an entity in a particular population of entities;

(e) estimating a joint probability that the values of the two or more selected descriptors of the first entity describe a second entity in the particular population of entities, different from the first entity, wherein the joint probability is estimated based, at least in part, on the individual probabilities;

(f) determining whether the joint probability of the values of the selected descriptors is less than a threshold probability;

(g) if the joint probability of the values of the selected descriptors is less than the threshold probability, searching in the plurality of data sets for a second data set including the values of the selected descriptors;

(h) if the second data set is found, performing an entity resolution operation, wherein performing the entity resolution operation comprises determining, based at least in part on the joint probability of the values of the selected descriptors being less than the threshold probability and on the first and second data sets including the values of the selected descriptors, that the second data set describes the first entity; and

repeating steps (b)-(h) one or more times,

wherein a false positive rate of the entity resolution operation is determined based on a size of the population and on the threshold probability, and

wherein estimating the joint probability that the values of all the selected descriptors describe a second entity in the particular population of entities comprises multiplying the individual probabilities of the values of the selected descriptors.

22. The storage medium of claim 21 , wherein determining the individual probability that the value of the descriptor describes an entity in the particular population of entities comprises counting occurrences of the descriptor value in a plurality of data sets representing entities in the population or in a sample of the population.

23. The storage medium of claim 21 , wherein the selected descriptors include first and second descriptors, wherein the values of the first and second descriptors are not independent, and wherein estimating the joint probability further comprises multiplying a product of the individual probabilities of the values of the first and second descriptors by a corrective factor.

24. The storage medium of claim 21 , wherein the operations further include setting the threshold probability based, at least in part, on the size of the population.

25. The storage medium of claim 24 , wherein setting the threshold probability based, at least in part, on the size of the population comprises setting the threshold probability to a value within one order of magnitude of the inverse of the size of the population.