Granted Patent B1

US 10,339,118 · App. 15/674,993 · Granted Jul 2, 2019

Inventor: Luke Davis (Winchcombe, GB)

Assignee: Palantir Technologies Inc.

G06F16/215G06F7/02G06F16/2468G06F16/2471G06F16/3329G06F16/3343G06F16/3344G06F16/3346

Abstract

Claims (61)

1. A data normalization system comprising:

one or more computer processors; and

one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, causes the data normalization system to perform operations comprising:

receiving a first string and a second string, the first string and the second string being ordered according to an initial string ordering;

converting, using a metaphone algorithm, the first string into a first metaphone string, and the second string into a second metaphone string;

searching, based on the first metaphone string and the second metaphone string, a name index including a listing of metaphone strings representing common names and probabilities that the common names are either a given name or a surname;

determining, based on searching the name index, a confidence score indicating a confidence level that the first string represents the given name and that the second string represents the surname;

determining that the confidence score does not meet or exceed a threshold confidence score;

in response to determining that the confidence score does not meet or exceed the threshold confidence score, analyzing, the first string and the second string based on a list of known character sets included in surnames, yielding an analysis;

determining, based on the analysis, that a set of characters in the second string matches a known character set included in the list of known character sets included in surnames; and

in response to determining that the set of characters in the second string matches a known character set included in the list of known character sets included in surname, ordering the first string and the second string according to an updated string ordering.

2. The data normalization system of claim 1 , the operations further comprising:

calculating an updated confidence score based on the analysis; and

determining that the updated confidence score meets or exceeds the threshold confidence score.

3. The data normalization system of claim 1 , the operations further comprising:

determining, based on searching the name index, a signal strength that the first metaphone string is a given name, a signal strength that the first metaphone string is a surname, a signal strength that the second metaphone string is a given name, and a signal strength that the second metaphone string is a surname;

determining an absolute difference between the signal strength that the first metaphone string is a surname and the signal strength that the first metaphone string is a given name, yielding an absolute signal difference for the first string;

determining an absolute difference between the signal strength that the second metaphone string is a surname and the signal strength that the second metaphone string is a given name, yielding an absolute signal difference for the second string; and

determining that the absolute signal difference for the first string is greater than the absolute signal difference for the second string.

4. The data normalization system of claim 3 , wherein the confidence score is further based on determining that the absolute signal difference for the first string is greater than the absolute signal difference for the second string.

5. The data normalization system of claim 1 wherein the name index includes at least a first set of metaphone strings corresponding to a first language set, and a second set of metaphone strings corresponding to a second language set.

6. The data normalization system of claim 5 , wherein a metaphone string from the first set of metaphone strings and a metaphone string from the second set of metaphone strings both represent an identical name.

7. A method comprising:

receiving a first string and a second string, the first string and the second string being ordered according to an initial string ordering;

converting, using a metaphone algorithm, the first string into a first metaphone string, and the second string into a second metaphone string;

searching, based on the first metaphone string and the second metaphone string, a name index including a listing of metaphone strings representing common names and probabilities that the common names are either a given name or a surname;

determining, based on searching the name index, a confidence score indicating a confidence level that the first string represents the given name and that the second string represents the surname;

determining that the confidence score does not meet or exceed a threshold confidence score;

in response to determining that the confidence score does not meet or exceed the threshold confidence score, analyzing, the first string and the second string based on a list of known character sets included in surnames, yielding an analysis;

determining, based on the analysis, that a set of characters in the second string matches a known character set included in the list of known character sets included in surnames; and

in response to determining that the set of characters in the second string matches a known character set included in the list of known character sets included in surname, ordering the first string and the second string according to an updated string ordering.

8. The method of claim 7 , further comprising:

calculating an updated confidence score based on the analysis; and

determining that the updated confidence score meets or exceeds the threshold confidence score.

9. The method of claim 7 , further comprising:

determining, based on searching the name index, a signal strength that the first metaphone string is a given name, a signal strength that the first metaphone string is a surname, a signal strength that the second metaphone string is a given name, and a signal strength that the second metaphone string is a surname;

determining an absolute difference between the signal strength that the first metaphone string is a surname and the signal strength that the first metaphone string is a given name, yielding an absolute signal difference for the first string;

determining an absolute difference between the signal strength that the second metaphone string is a surname and the signal strength that the second metaphone string is a given name, yielding an absolute signal difference for the second string; and

determining that the absolute signal difference for the first string is greater than the absolute signal difference for the second string.

10. The method of claim 9 , wherein the confidence score is further based on determining that the absolute signal difference for the first string is greater than the absolute signal difference for the second string.

11. The method of claim 7 , wherein the name index includes at least a first set of metaphone strings corresponding to a first language set, and a second set of metaphone strings corresponding to a second language set.

12. The method of claim 11 , wherein a metaphone string from the first set of metaphone strings and a metaphone string from the second set of metaphone strings both represent an identical name.

13. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a data normalization system, causes the data normalization system to perform operations comprising:

receiving a first string and a second string, the first string and the second string being ordered according to an initial string ordering;

converting, using a metaphone algorithm, the first string into a first metaphone string, and the second string into a second metaphone string;

searching, based on the first metaphone string and the second metaphone string, a name index including a listing of metaphone strings representing common names and probabilities that the common names are either a given name or a surname;

determining, based on searching the name index, a confidence score indicating a confidence level that the first string represents the given name and that the second string represents the surname;

determining that the confidence score does not meet or exceed a threshold confidence score;

in response to determining that the confidence score does not meet or exceed the threshold confidence score, analyzing, the first string and the second string based on a list of known character sets included in surnames, yielding an analysis;

determining, based on the analysis, that a set of characters in the second string matches a known character set included in the list of known character sets included in surnames; and

in response to determining that the set of characters in the second string matches a known character set included in the list of known character sets included in surname, ordering the first string and the second string according to an updated string ordering.

14. The non-transitory computer-readable medium of claim 13 , the operations further comprising:

calculating an updated confidence score based on the analysis; and

determining that the updated confidence score meets or exceeds the threshold confidence score.

15. The non-transitory computer-readable medium of claim 13 , the operations further comprising:

determining, based on searching the name index, a signal strength that the first metaphone string is a given name, a signal strength that the first metaphone string is a surname, a signal strength that the second metaphone string is a given name, and a signal strength that the second metaphone string is a surname;

determining an absolute difference between the signal strength that the first metaphone string is a surname and the signal strength that the first metaphone string is a given name, yielding an absolute signal difference for the first string;

determining an absolute difference between the signal strength that the second metaphone string is a surname and the signal strength that the second metaphone string is a given name, yielding an absolute signal difference for the second string; and

determining that the absolute signal difference for the first string is greater than the absolute signal difference for the second string.

16. The non-transitory computer-readable medium of claim 15 , wherein the confidence score is further based on determining that the absolute signal difference for the first string is greater than the absolute signal difference for the second string.

17. The non-transitory computer-readable medium of claim 13 , wherein the name index includes at least a first set of metaphone strings corresponding to a first language set, and a second set of metaphone strings corresponding to a second language set, and a metaphone string from the first set of metaphone strings and a metaphone string from the second set of metaphone strings both represent an identical name.

Assignments (8)

Continuity (1)