Systems and methods for deterministic error detection functions in large data structures

1 . A networked database system including a software application for operation on a user device, the system comprising:

a processor configured for manipulation of health records associated with a user;

a tangible, non-transitory electronic memory in electronic communication with the processor,

the tangible, non-transitory memory having instructions stored thereon that, in response to execution by the processor, cause the processor to perform operations comprising:

storing, by the processor, a plurality of records;

applying, by the processor, a first hash function to a respective first name initial, a respective first name character length, a respective last name initial, and respective a last name character length of each respective record in the plurality of records;

applying, by the processor, a second hash function to a respective first two characters of a respective first name, a respective first two characters of a respective last name, and a combined character length of the respective first name and the respective last name for each respective record in the plurality of records;

generating, by the processor and based on the first hash function and the second hash function, a hash index;

identifying, by the processor, matching hashes within the hash index;

flagging, by the processor, similar records associated with the matching hashes of the hash index; and

merging the flagged records.

2 . The system of claim 1 , wherein the first hash function and the second hash function return an eight digit integer.

3 . The system of claim 2 , wherein first and second digits of the eight digit integer encode the respective first name initial, third and fourth digits of the eight digit integer encode the respective last name initial, and remining digits of the eight digit integer encode the combined character length of the respective first name and the respective last name.

4 . The system of claim 1 , wherein:

the identifying the matching hashes comprises generating, using a machine learning algorithm, a respective similarity score between records of the plurality of records; and

the flagging the similar records comprises, when the similarity score is above a predetermined level, flagging the similar records associated with the matching hashes of the hash index.

5 . The system of claim 4 , wherein using the machine learning algorithm comprises:

scanning rows of the hash index;

generating a cluster of rows; and

generating the similarity score between rows within the cluster of rows.

6 . The system of claim 4 , wherein the instructions cause the processor to perform further operations comprising training the machine learning algorithm on a training data set of health records that is different from the plurality of records.

7 . The system of claim 1 , wherein the instructions cause the processor to perform further operations comprising:

identifying, by the processor, non-matching hashes within the hash index;

flagging, by the processor, dissimilar records associated with the non-matching hashes of the hash index; and

storing the dissimilar records as new records.

8 . A method comprising:

storing, by a processor, a plurality of records;

applying, by the processor, a first hash function to a respective first name initial, a respective first name character length, a respective last name initial, and respective a last name character length of each respective record in the plurality of records;

applying, by the processor, a second hash function to a respective first two characters of a respective first name, a respective first two characters of a respective last name, and a combined character length of the respective first name and the respective last name for each respective record in the plurality of records;

generating, by the processor and based on the first hash function and the second hash function, a hash index wherein each hash of the hash index is associated on a one-to-one basis with each respective record of the plurality of records;

identifying, by the processor, matching hashes within the hash index;

flagging, by the processor, similar records associated with the matching hashes of the hash index; and

merging the flagged records.

9 . The method of claim 8 , wherein the first hash function and the second hash function return an eight digit integer.

10 . The method of claim 9 , wherein first and second digits of the eight digit integer encode the respective first name initial, third and fourth digits of the eight digit integer encode the respective last name initial, and remining digits of the eight digit integer encode the combined character length of the respective first name and the respective last name.

11 . The method of claim 8 , wherein:

the identifying the matching hashes comprises generating, using a machine learning algorithm, a respective similarity score between records of the plurality of records; and

the flagging the similar records comprises, when the similarity score is above a predetermined level, flagging the similar records associated with the matching hashes of the hash index.

12 . The method of claim 11 , wherein using the machine learning algorithm comprises:

scanning rows of the hash index;

generating a cluster of rows; and

generating the similarity score between rows within the cluster of rows.

13 . The method of claim 11 further comprising training the machine learning algorithm on a training data set of health records that is different from the plurality of records.

14 . The method of claim 8 further comprising:

identifying, by the processor, non-matching hashes within the hash index;

flagging, by the processor, dissimilar records associated with the non-matching hashes of the hash index; and

storing the dissimilar records as new records.

15 . An article of manufacture including a non-transitory, tangible computer readable storage medium having instructions stored thereon that, in response to execution by a processor, cause the processor to perform operations comprising:

storing, by the processor, a plurality of records;

applying, by the processor, a first hash function to a respective first name initial, a respective first name character length, a respective last name initial, and respective a last name character length of each respective record in the plurality of records;

applying, by the processor, a second hash function to a respective first two characters of a respective first name, a respective first two characters of a respective last name, and a combined character length of the respective first name and the respective last name for each respective record in the plurality of records;

generating, by the processor and based on the first hash function and the second hash function, a hash index wherein each hash of the hash index is associated on a one-to-one basis with each respective record of the plurality of records;

identifying, by the processor, matching hashes within the hash index;

flagging, by the processor, similar records associated with the matching hashes of the hash index; and

merging the flagged records.

16 . The article of manufacture of claim 15 , wherein the first hash function and the second hash function return an eight digit integer.

17 . The article of manufacture of claim 16 , wherein first and second digits of the eight digit integer encode the respective first name initial, third and fourth digits of the eight digit integer encode the respective last name initial, and remining digits of the eight digit integer encode the combined character length of the respective first name and the respective last name.

18 . The article of manufacture of claim 15 , wherein:

the identifying the matching hashes comprises generating, using a machine learning algorithm, a respective similarity score between records of the plurality of records; and

the flagging the similar records comprises, when the similarity score is above a predetermined level, flagging the similar records associated with the matching hashes of the hash index.

19 . The article of manufacture of claim 18 , wherein using the machine learning algorithm comprises:

scanning rows of the hash index;

generating a cluster of rows; and

generating the similarity score between rows within the cluster of rows.

20 . The article of manufacture of claim 18 , wherein the instructions cause the processor to perform further operations comprising training the machine learning algorithm on a training data set of health records that is different from the plurality of records.