Synthetic data generation using anonymity preservation in computer-based reasoning systems
Techniques for synthetic data generation in computer-based reasoning systems are discussed and include receiving a request for generation of synthetic data based on a set of training data cases. One or more focal training data cases are determined. For undetermined features (either all of them or those that are not subject to conditions), a value for the feature is determined based on the focal cases. In some embodiments, the generated synthetic data may be checked for similarity against the original data, and if similarity conditions are met, it may be modified (e.g., resampled), removed, and/or replaced.
1. A non-transitory computer readable medium storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform a process of:
receiving a request for generation of synthetic data based on a set of training data cases;
determining one or more focal training data cases from among the set of training data cases;
determining a synthetic data case based on the one or more focal training data cases;
determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases;
when the synthetic data case is determined to be overly similar to one or more cases in the set of training data cases, taking corrective action for the synthetic data case to produce a new synthetic data case and using the new synthetic data case for consideration in adding to a set of synthetic training data cases;
when the synthetic data case is determined to not be overly similar to cases in the set of training data cases, adding the synthetic data case to the set of synthetic training data cases.
2. The non-transitory computer readable medium of claim 1 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a prediction conviction score for the synthetic data case with respect to the set of training data cases, and when the prediction conviction score for the synthetic data case is beyond a threshold, taking corrective action for the synthetic data case to produce a new synthetic data case, and wherein the prediction conviction score can be expressed as information required to describe a position of a point in question relative to existing points.
3. The non-transitory computer readable medium of claim 1 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a certainty score for the synthetic data case with respect to the set of training data cases, and when the certainty score for the synthetic data case is beyond a threshold, taking corrective action for the synthetic data case to produce a new synthetic data case.
4. The non-transitory computer readable medium of claim 3 , wherein the certainty score is determined based on a familiarity conviction score, wherein the familiarity conviction score is a measure of how much information the point distorts the model.
5. The non-transitory computer readable medium of claim 1 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a prediction conviction score for the synthetic data case with respect to the set of training data cases, and when the prediction conviction score for the synthetic data case is beyond a threshold, determining a probability of taking corrective action for the synthetic data case to produce a new synthetic data case based on the prediction conviction score, and determining whether to take correction action based on the probability of taking corrective action for the synthetic data case, wherein taking correcting action for the synthetic data case includes producing a new synthetic data case, and wherein the prediction conviction score can be expressed as information required to describe a position of a point in question relative to existing points.
6. The non-transitory computer readable medium of claim 1 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a certainty score for the synthetic data case with respect to the set of training data cases, and when the certainty score for the synthetic data case is beyond a threshold, determining a probability of taking corrective action for the synthetic data case to produce a new synthetic data case based on the certainty score, and determining whether to take correction action based on the probability of taking corrective action for the synthetic data case, wherein taking correcting action for the synthetic data case includes producing a new synthetic data case.
7. The non-transitory computer readable medium of claim 6 , wherein the certainty score is determined based on a familiarity conviction score, wherein the familiarity conviction score is a measure of how much information a point distorts the model.
8. The non-transitory computer readable medium of claim 1 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining whether the synthetic data case is overly similar to only one or more cases comprising a subset of the features of the cases in the set of training data cases.
9. The non-transitory computer readable medium of claim 1 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining whether the synthetic data case is overly similar to only one or more cases comprising a subset of the cases in the set of training data cases.
10. The non-transitory computer readable medium of claim 1 , wherein taking corrective action comprises one or more of
modify the synthetic data case to produce the new synthetic data case or
replacing the synthetic data case with a newly-generated synthetic data case.
11. A system for executing instructions, wherein said instructions are instructions which, when executed by one or more computing devices, cause performance of a process including:
receiving a request for generation of synthetic data based on a set of training data cases;
determining one or more focal training data cases from among the set of training data cases;
determining a synthetic data case based on the one or more focal training data cases;
determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases;
when the synthetic data case is determined to be overly similar to one or more cases in the set of training data cases, taking corrective action for the synthetic data case to produce a new synthetic data case and using the new synthetic data case for consideration in adding to a set of synthetic training data cases;
when the synthetic data case is determined to not be overly similar to cases in the set of training data cases, adding the synthetic data case to the set of synthetic training data cases.
12. The system of claim 11 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a prediction conviction score for the synthetic data case with respect to the set of training data cases, and when the prediction conviction score for the synthetic data case is beyond a threshold, taking corrective action for the synthetic data case to produce a new synthetic data case, and wherein the prediction conviction score can be expressed as information required to describe a position of a point in question relative to existing points.
13. The system of claim 11 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a certainty score for the synthetic data case with respect to the set of training data cases, and when the certainty score for the synthetic data case is beyond a threshold, taking corrective action for the synthetic data case to produce a new synthetic data case.
14. The system of claim 13 , wherein the certainty score is determined based on a familiarity conviction score, wherein the familiarity conviction score is a measure of how much information the point distorts the model.
15. The system of claim 11 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a prediction conviction score for the synthetic data case with respect to the set of training data cases, and when the prediction conviction score for the synthetic data case is beyond a threshold, determining a probability of taking corrective action for the synthetic data case to produce a new synthetic data case based on the prediction conviction score, and determining whether to take correction action based on the probability of taking corrective action for the synthetic data case, wherein taking correcting action for the synthetic data case includes producing a new synthetic data case, and wherein the prediction conviction score can be expressed as information required to describe a position of a point in question relative to existing points.
16. A method comprising:
receiving a request for generation of synthetic data based on a set of training data cases;
determining one or more focal training data cases from among the set of training data cases
determining a synthetic data case based on the one or more focal training data cases;
determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases;
when the synthetic data case is determined to be overly similar to one or more cases in the set of training data cases, taking corrective action for the synthetic data case to produce a new synthetic data case and using the new synthetic data case for consideration in adding to a set of synthetic training data cases;
when the synthetic data case is determined to not be overly similar to cases in the set of training data cases, adding the synthetic data case to the set of synthetic training data cases,
wherein the method is performed on one or more computing devices.
17. The method of claim 16 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a prediction conviction score for the synthetic data case with respect to the set of training data cases, and when the prediction conviction score for the synthetic data case is beyond a threshold, taking corrective action for the synthetic data case to produce a new synthetic data case, and wherein the prediction conviction score can be expressed as information required to describe a position of a point in question relative to existing points.
18. The method of claim 16 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a certainty score for the synthetic data case with respect to the set of training data cases, and when the certainty score for the synthetic data case is beyond a threshold, taking corrective action for the synthetic data case to produce a new synthetic data case.
19. The method of claim 18 , wherein the certainty score is determined based on a familiarity conviction score, wherein the familiarity conviction score is a measure of how much information the point distorts the model.
20. The method of claim 16 , wherein determining whether the synthetic data case is overly similar to one or more cases in the set of training data cases comprises determining a prediction conviction score for the synthetic data case with respect to the set of training data cases, and when the prediction conviction score for the synthetic data case is beyond a threshold, determining a probability of taking corrective action for the synthetic data case to produce a new synthetic data case based on the prediction conviction score, and determining whether to take correction action based on the probability of taking corrective action for the synthetic data case, wherein taking correcting action for the synthetic data case includes producing a new synthetic data case, and wherein the prediction conviction score can be expressed as information required to describe a position of a point in question relative to existing points.