Systems and/or methods for automated repair of data used in machine learning models via counterfactual generation and mapping

1 . A system for repairing a dataset to be used in a downstream machine learning (ML) model, the system comprising:

an interface to a data store storing the dataset; and

a memory and at least one processor configured to perform operations comprising:

extracting features from the dataset stored to the data store, on a portion-by-portion basis; and

for each portion:

applying an error detection algorithm to identify erroneous data instances in the dataset using the extracted features associated with the respective portion; and

for each identified erroneous data instance:

determining a set of feature-space repair candidates including a plurality of feature-space repair candidates, each feature-space repair candidate in the set of feature-space repair candidates having at least one change with respect to the extracted features associated with the respective erroneous data instance;

for each feature-space repair candidate, generating a data-space repair candidate by (a) finding a matching non-erroneous data instance within the respective portion, and (b) using one or more values from the matching non-erroneous data instance in place of one or more corresponding values in the respective erroneous data instance;

generating a repaired instance based on one or more of the data-space repair candidates; and

updating the portion to include the repaired instance rather than the respective erroneous data instance, in repairing the dataset in the data store.

2 . The system of claim 1 , wherein the extracted features include metadata, one or more semantic embeddings, and one or more statistical representations related to the dataset in the data store.

3 . The system of claim 1 , wherein the error detection algorithm is an ML-based error detector.

4 . The system of claim 1 , wherein a plurality of error detection algorithms are applied in the identification of the erroneous data instances in the dataset.

5 . The system of claim 1 , wherein the application of the error detection algorithm assigns a label to each data instance in each portion, the label identifying whether the associated data instance is an erroneous data instance or a clean data instance.

6 . The system of claim 1 , wherein matching non-erroneous data instances are located by finding a nearest neighbor based on a similarity metric.

7 . The system of claim 1 , wherein the generation of a given repaired instance based on one or more corresponding data-space repair candidates comprises assigning a weight to each corresponding data-space repair candidate, the weight indicating a potential contribution of the associated data-space repair candidate to performance of the ML model.

8 . The system of claim 7 , wherein the generation of the given repaired instance further comprises:

assigning a weight to each corresponding data-space repair candidate, the weight indicating a potential contribution of the associated data-space repair candidate to performance of the ML model;

in response to the assigned weights indicating that there is a single best corresponding data-space repair candidate, using the single best corresponding data-space repair candidate as the given repaired instance; and

in response to the assigned weights indicating that there is not a single best corresponding data-space repair candidate, combining the corresponding data-space repair candidates in accordance with their assigned weights in generating the given repaired instance.

9 . The system of claim 1 , wherein the feature-space repair candidates are generated as counterfactuals, wherein each counterfactual represents a minimal set of changes to the feature(s) of the associated erroneous data instance that would change output from the error detection algorithm from erroneous to clean.

10 . A method for repairing a dataset to be used in a downstream machine learning (ML) model, the method comprising:

extracting features from a dataset, which is stored in a data store, on a portion-by-portion basis; and

for each portion:

applying an error detection algorithm to identify erroneous data instances in the dataset using the extracted features associated with the respective portion; and

for each identified erroneous data instance:

determining a set of feature-space repair candidates including a plurality of feature-space repair candidates, each feature-space repair candidate in the set of feature-space repair candidates having at least one change with respect to the extracted features associated with the respective erroneous data instance;

for each feature-space repair candidate, generating a data-space repair candidate by (a) finding a matching non-erroneous data instance within the respective portion, and (b) using one or more values from the matching non-erroneous data instance in place of one or more corresponding values in the respective erroneous data instance;

generating a repaired instance based on one or more of the data-space repair candidates; and

updating the portion to include the repaired instance rather than the respective erroneous data instance, in repairing the dataset in the data store.

11 . The method of claim 10 , wherein the extracted features include metadata, one or more semantic embeddings, and one or more statistical representations related to the dataset in the data store.

12 . The method of claim 10 , wherein the application of the error detection algorithm assigns a label to each data instance in each portion, the label identifying whether the associated data instance is an erroneous data instance or a clean data instance.

13 . The method of claim 10 , wherein matching non-erroneous data instances are located by finding a nearest neighbor based on a similarity metric.

14 . The method of claim 10 , wherein the generation of a given repaired instance based on one or more corresponding data-space repair candidates comprises:

assigning a weight to each corresponding data-space repair candidate, the weight indicating a potential contribution of the associated data-space repair candidate to performance of the ML model;

assigning a weight to each corresponding data-space repair candidate, the weight indicating a potential contribution of the associated data-space repair candidate to performance of the ML model;

in response to the assigned weights indicating that there is a single best corresponding data-space repair candidate, using the single best corresponding data-space repair candidate as the given repaired instance; and

in response to the assigned weights indicating that there is not a single best corresponding data-space repair candidate, combining the corresponding data-space repair candidates in accordance with their assigned weights in generating the given repaired instance.

15 . The method of claim 10 , wherein the feature-space repair candidates are generated as counterfactuals, such that wherein each counterfactual represents a minimal set of changes to the feature(s) of the associated erroneous data instance that would change output from the error detection algorithm from erroneous to clean.

16 . A non-transitory computer readable storage medium tangibly storing instructions that, when executed by at least one processor, perform operations comprising:

extracting features on a portion-by-portion basis from a dataset, which is stored in a data store, to be used in a downstream machine learning (ML) model; and

for each portion:

applying an error detection algorithm to identify erroneous data instances in the dataset using the extracted features associated with the respective portion; and

for each identified erroneous data instance:

determining a set of feature-space repair candidates including a plurality of feature-space repair candidates, each feature-space repair candidate in the set of feature-space repair candidates having at least one change with respect to the extracted features associated with the respective erroneous data instance;

for each feature-space repair candidate, generating a data-space repair candidate by (a) finding a matching non-erroneous data instance within the respective portion, and (b) using one or more values from the matching non-erroneous data instance in place of in Hey-of-one or more corresponding values in the respective erroneous data instance;

generating a repaired instance based on one or more of the data-space repair candidates; and

updating the portion to include the repaired instance rather than the respective erroneous data instance, in repairing the dataset in the data store.

17 . The non-transitory computer readable storage medium of claim 16 , wherein the extracted features include metadata, one or more semantic embeddings, and one or more statistical representations related to the dataset in the data store.

18 . The non-transitory computer readable storage medium of claim 16 , wherein the application of the error detection algorithm assigns a label to each data instance in each portion, the label identifying whether the associated data instance is an erroneous data instance or a clean data instance.

19 . The non-transitory computer readable storage medium of claim 16 , wherein the generation of a given repaired instance based on one or more corresponding data-space repair candidates comprises:

assigning a weight to each corresponding data-space repair candidate, the weight indicating a potential contribution of the associated data-space repair candidate to performance of the ML model;

assigning a weight to each corresponding data-space repair candidate, the weight indicating a potential contribution of the associated data-space repair candidate to performance of the ML model;

in response to the assigned weights indicating that there is a single best corresponding data-space repair candidate, using the single best corresponding data-space repair candidate as the given repaired instance; and

in response to the assigned weights indicating that there is not a single best corresponding data-space repair candidate, combining the corresponding data-space repair candidates in accordance with their assigned weights in generating the given repaired instance.

20 . The non-transitory computer readable storage medium of claim 16 , wherein the feature-space repair candidates are generated as counterfactuals, such wherein each counterfactual represents a minimal set of changes to the feature(s) of the associated erroneous data instance that would change output from the error detection algorithm from erroneous to clean.