Match index creation

1. A system for match index creation, the system comprising:

one or more processors; and

a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:

identify, by a database system, a first number of distinct values stored in a first field by a dataset of records;

identify, by the database system, a second number of distinct values stored in a second field by the dataset of records;

determine, by the database system, whether the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field;

create, by the database system, a trie from values stored in the first field by the dataset of records, in response to a determination that the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field;

associate, by the database system, based on a value stored in the first field by a record of the dataset of records, a node in the trie with the record;

identify, by the database system, based on a prospective value stored in the first field by the prospective record, a branch sequence in the trie as a match key for a prospective record; and

identify, by the database system, using the match key for the prospective record, a record of the dataset of records that matches the prospective record.

2. The system of claim 1 , wherein the distinct values comprise distinct prefixes of text values, each distinct prefix being associated with a number of records of the dataset of records, the number of records being less than a record threshold.

3. The system of claim 1 , wherein determining whether the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field is based on counts of each distinct value stored in the first field and counts of each distinct value stored in the second field.

4. The system of claim 1 , wherein determining whether the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field is based on a count of numeric bins associated with values stored in the first field and a count of numeric bins associated with values stored in the second field.

5. The system of claim 1 , wherein creating the trie comprises:

tokenizing, by the database system, the values stored in the first field by the dataset of records; and

creating, by the database system, the trie from the tokenized values, each branch in the trie labeled with one of the tokenized values, each node storing a count indicating a number of the dataset of records associated with a tokenized value sequence beginning from a root of the trie.

6. The system of claim 1 , wherein associating the node in the trie with the record comprises:

tokenizing, by the database system, the value stored in the first field by the record;

identifying, by the database system, each node, beginning from a root of the trie, corresponding to a token value sequence associated with the tokenized value, until a node is identified that stores a count less than a node threshold;

identifying, by the database system, a branch sequence comprising each identified node as a match key for the record; and

associating, by the database system, the match key with the node, and the record with the match key.

7. The system of claim 1 , wherein identifying the branch sequence as the match key for the prospective record comprises:

tokenizing, by the database system, the prospective value stored in the first field by the prospective record;

identifying, by the database system, each node, beginning from a root of the trie, corresponding to a token value sequence associated with the tokenized prospective value, until another node is identified that stores a count that is less than a node threshold; and

identifying, by the database system, a match key associated with the other node as the match key for the prospective record.

8. A computer program product comprising computer-readable program code to be executed by one or more processors when retrieved from a non-transitory computer-readable medium, the program code including instructions to:

identify, by a database system, a first number of distinct values stored in a first field by a dataset of records;

identify, by the database system, a second number of distinct values stored in a second field by the dataset of records;

determine, by the database system, whether the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field;

create, by the database system, a trie from values stored in the first field by the dataset of records, in response to a determination that the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field;

associate, by the database system, based on a value stored in the first field by a record of the dataset of records, a node in the trie with the record;

identify, by the database system, based on a prospective value stored in the first field by the prospective record, a branch sequence in the trie as a match key for a prospective record; and

identify, by the database system, using the match key for the prospective record, a record of the dataset of records that matches the prospective record.

9. The computer program product of claim 8 , wherein the distinct values comprise distinct prefixes of text values, each distinct prefix being associated with a number of records of the dataset of records, the number of records being less than a record threshold.

10. The computer program product of claim 8 , wherein determining whether the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field is based on counts of each distinct value stored in the first field and counts of each distinct value stored in the second field.

11. The computer program product of claim 8 , wherein determining whether the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field is based on a count of numeric bins associated with values stored in the first field and a count of numeric bins associated with values stored in the second field.

12. The computer program product of claim 8 , wherein creating the trie comprises:

tokenizing, by the database system, the values stored in the first field by the dataset of records; and

creating, by the database system, the trie from the tokenized values, each branch in the trie labeled with one of the tokenized values, each node storing a count indicating a number of the dataset of records associated with a tokenized value sequence beginning from a root of the trie.

13. The computer program product of claim 8 , wherein associating the node in the trie with the record comprises:

tokenizing, by the database system, the value stored in the first field by the record;

identifying, by the database system, each node, beginning from a root of the trie, corresponding to a token value sequence associated with the tokenized value, until a node is identified that stores a count less than a node threshold;

identifying, by the database system, a branch sequence comprising each identified node as a match key for the record; and

associating, by the database system, the match key with the node, and the record with the match key.

14. The computer program product of claim 8 , wherein identifying the branch sequence as the match key for the prospective record comprises:

tokenizing, by the database system, the prospective value stored in the first field by the prospective record;

identifying, by the database system, each node, beginning from a root of the trie, corresponding to a token value sequence associated with the tokenized prospective value, until another node is identified that stores a count that is less than a node threshold; and

identifying, by the database system, a match key associated with the other node as the match key for the prospective record.

15. A method comprising:

identifying, by a database system, a first number of distinct values stored in a first field by a dataset of records;

identifying, by the database system, a second number of distinct values stored in a second field by the dataset of records;

determining, by the database system, whether the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field;

creating by the database system, a trie from values stored in the first field by the dataset of records, in response to a determination that the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field;

associating, by the database system, based on a value stored in the first field by a record of the dataset of records, a node in the trie with the record;

identifying, by the database system, based on a prospective value stored in the first field by the prospective record, a branch sequence in the trie as a match key for a prospective record; and

identifying, by the database system, using the match key for the prospective record, a record of the dataset of records that matches the prospective record.

16. The method of claim 15 , wherein the distinct values comprise distinct prefixes of text values, each distinct prefix being associated with a number of records of the dataset of records, the number of records being less than a record threshold.

17. The method of claim 15 , wherein determining whether the first number of distinct values stored in the first field is greater than the second number of distinct values stored in the second field is based on at least one of counts of each distinct value stored in the first field and counts of each distinct value stored in the second field, and a count of numeric bins associated with values stored in the first field and a count of numeric bins associated with values stored in the second field.

18. The method of claim 15 , wherein creating the trie comprises:

tokenizing, by the database system, the values stored in the first field by the dataset of records; and

creating, by the database system, the trie from the tokenized values, each branch in the trie labeled with one of the tokenized values, each node storing a count indicating a number of the dataset of records associated with a tokenized value sequence beginning from a root of the trie.

19. The method of claim 15 , wherein associating the node in the trie with the record comprises:

tokenizing, by the database system, the value stored in the first field by the record;

identifying, by the database system, each node, beginning from a root of the trie, corresponding to a token value sequence associated with the tokenized value, until a node is identified that stores a count less than a node threshold;

identifying, by the database system, a branch sequence comprising each identified node as a match key for the record; and

associating, by the database system, the match key with the node, and the record with the match key.

20. The method of claim 15 , wherein identifying the branch sequence as the match key for the prospective record comprises:

tokenizing, by the database system, the prospective value stored in the first field by the prospective record;

identifying, by the database system, each node, beginning from a root of the trie, corresponding to a token value sequence associated with the tokenized prospective value, until another node is identified that stores a count that is less than a node threshold; and

identifying, by the database system, a match key associated with the other node as the match key for the prospective record.