Entity resolution framework for data matching

1. A data storage and retrieval method for matching a corrupted database record with a record of a validated database using a denoising autoencoder, the method comprising:

receiving a first corrupted record from a first database, the first corrupted record comprising a first plurality of data fields;

generating a first input data vector based on the first corrupted record;

selecting a denoising autoencoder specific to the first database, wherein the denoising autoencoder was trained using a plurality of training example data pairs;

applying the selected denoising autoencoder to the first input data vector to generate a first denoised data vector;

comparing the first denoised data vector with each of a plurality of validated data vectors generated based on records of the validated database to determine whether the first denoised data vector matches a first matching vector of the plurality of data vectors; and

in response to determining that the first denoised data vector matches the first matching vector:

providing a data pair comprising the first input data vector and the first matching vector as an additional training example data pair to the denoising autoencoder to train the denoising autoencoder to denoise input from the first database;

retrieving, from the validated database, a first validated record that was used to generate the first matching vector; and

outputting the retrieved first validated record.

2. The method of claim 1 , the method further comprising:

receiving a second corrupted record from a first database, the second corrupted record comprising a second plurality of data fields;

generating a second input data vector based on the second corrupted record;

accessing a denoising autoencoder specific to the first database, wherein the denoising autoencoder was trained using the data pair comprising the first input data vector and the first matching vector;

generating a second denoised data vector by applying the denoising autoencoder to the second corrupted data vector;

comparing the second denoised data vector to the plurality of validated data vectors to determine whether the second denoised data vector matches a second matching vector of the plurality of data vectors; and

in response to determining that the second denoised data vector matches the second matching vector:

retrieving, from the validated database, a second validated record that was used to generate the second matching vector; and

outputting the retrieved second validated record.

3. The method of claim 1 , wherein generating the first input data vector based on the first corrupted record comprises:

converting each field of the first plurality of data fields into binary notation; and

concatenating the converted data fields.

4. The method of claim 1 , wherein the denoising autoencoder specific to the first database comprises a neural network that comprises an input layer, a hidden layer, and an output layer, and wherein the hidden layer includes fewer neurons than the input layer.

5. The method of claim 4 , wherein the hidden layer includes fewer neurons than the output layer.

6. The method of claim 4 , wherein training the denoising autoencoder to denoise input from the first database comprises:

adjusting neuron connections between neurons of the neural network based on the example data pairs.

7. The method of claim 6 , wherein adjusting neuron connections between neurons of the neural network based on the example data pairs comprises:

creating a first plurality of copies of the neural network, wherein at least some of the neuron connections of each copy of the neural network is randomized;

calculating a performance score for each copy of the neural network for the example data pairs;

deleting copies of the neural network that have a performance score that is below a threshold; and

generating a second plurality of copies of the neural network based on a portion of the copies of the neural network that have performed well.

8. The method of claim 1 , further comprising transmitting the retrieved first validated record to the first database to replace the first corrupted record in storage of the first database.

9. The method of claim 1 , wherein selecting the denoising autoencoder specific to the first database comprises:

accessing data of the denoising autoencoder to extract database identification data; and

selecting the denoising autoencoder when the extracted database identification data matches identification data of the first database.

10. The method of claim 1 , wherein comparing the first denoised data vector with a particular validated data vector of the plurality of validated data vectors comprises:

performing binary pairwise comparisons between elements of the data vector and elements of the particular validated data vector; and

calculating a percentage of pairwise comparisons that resulted in a match.

11. A data storage and retrieval system for matching a corrupted database record with a record of a validated database using a denoising autoencoder, the system comprising:

input circuitry of a device configured to receive a first corrupted record from a first database, the first corrupted record comprising a first plurality of data fields; and

control circuitry of the device configured to:

generate a first input data vector based on the first corrupted record;

select a denoising autoencoder specific to the first database, wherein the denoising autoencoder was trained using a plurality of training example data pairs;

apply the selected denoising autoencoder to the first input data vector to generate a first denoised data vector;

compare the first denoised data vector with each of a plurality of validated data vectors generated based on records of the validated database to determine whether the first denoised data vector matches a first matching vector of the plurality of data vectors; and

in response to determining that the first denoised data vector matches the first matching vector:

provide a data pair comprising the first input data vector and the first matching vector as an additional training example data pair to the denoising autoencoder to train the denoising autoencoder to denoise input from the first database;

retrieve, from the validated database, a first validated record that was used to generate the first matching vector; and

output the retrieved first validated record.

12. The system of claim 11 , wherein the control circuitry is further configured to:

receive a second corrupted record from a first database, the second corrupted record comprising a second plurality of data fields;

generate a second input data vector based on the second corrupted record;

access a denoising autoencoder specific to the first database, wherein the denoising autoencoder was trained using the data pair comprising the first input data vector and the first matching vector;

generate a second denoised data vector by applying the denoising autoencoder to the second corrupted data vector;

compare the second denoised data vector to the plurality of validated data vectors to determine whether the second denoised data vector matches a second matching vector of the plurality of data vectors; and

in response to determining that the second denoised data vector matches the second matching vector:

retrieve, from the validated database, a second validated record that was used to generate the second matching vector; and

output the retrieved second validated record.

13. The system of claim 11 , wherein, the control circuitry is configured to generate the first input data vector based on the first corrupted record by:

converting each field of the first plurality of data fields into binary notation; and

concatenating the converted data fields.

14. The system of claim 11 , wherein the denoising autoencoder specific to the first database comprises a neural network that comprises an input layer, a hidden layer, and an output layer, and wherein the hidden layer includes fewer neurons than the input layer.

15. The system of claim 14 , wherein the hidden layer includes fewer neurons than the output layer.

16. The system of claim 14 , wherein, the control circuitry is configured to train the denoising autoencoder to denoise input from the first database by:

adjusting neuron connections between neurons of the neural network based on the example data pairs.

17. The system of claim 16 , wherein, the control circuitry is configured to adjust neuron connections between neurons of the neural network based on the example data pairs by:

creating a first plurality of copies of the neural network, wherein at least some of the neuron connections of each copy of the neural network is randomized;

calculating a performance score for each copy of the neural network for the example data pairs;

deleting copies of the neural network that have a performance score that is below a threshold; and

generating a second plurality of copies of the neural network based on a portion of the copies of the neural network that have performed well.

18. The system of claim 11 , wherein the control circuitry is configured to transmit the retrieved first validated record to the first database to replace the first corrupted record in storage of the first database.

19. The system of claim 11 , wherein the control circuitry is configured to select the denoising autoencoder specific to the first database by:

accessing data of the denoising autoencoder to extract database identification data; and

selecting the denoising autoencoder when the extracted database identification data matches identification data of the first database.

20. The system of claim 11 , wherein the control circuitry is configured to select the denoising autoencoder specific to the first database by:

performing binary pairwise comparisons between elements of the data vector and elements of the particular validated data vector; and

calculating a percentage of pairwise comparisons that resulted in a match.