Identifier contribution allocation in synthetic data generation in computer-based reasoning systems
Techniques for synthetic data generation in computer-based reasoning systems are discussed and include receiving a request for generation of synthetic data based on a set of training data cases. One or more focal training data cases are determined. For undetermined features (either all of them or those that are not subject to conditions), a value for the feature is determined based on the focal cases. In some embodiments, the generated synthetic data may be checked for similarity against the training data, and if similarity conditions are met, it may be modified (e.g., resampled), removed, and/or replaced.
1. A method comprising:
receiving a request for generation of synthetic data based on a set of training data cases;
for each synthetic data case in the synthetic data,
determining a first undetermined feature in the synthetic data case by:
determining one or more focal training data cases from among the set of training data cases based at least in part on one or more conditions;
determining a value for the first undetermined feature in the synthetic data case based at least in part on the one or more focal training data cases;
using the value for the first undetermined feature in the synthetic data case;
determining subsequent undetermined features in the synthetic data case by:
determining one or more focal training data cases from among the set of training data cases based at least in part on the value for the first undetermined feature and any previously-determined values for subsequent undetermined features;
determining a value for the subsequent undetermined feature in the synthetic data case based at least in part on the one or more focal training data cases;
using the value for the subsequent undetermined feature in the synthetic data case;
continuing to determine subsequent undetermined features until there are no more undetermined features in the synthetic data case;
causing control of a controllable system using a computer-based reasoning model that was determined at least in part based on the synthetic data cases in the synthetic data;
wherein the method is performed by one or more computing devices.
2. The method of claim 1 , wherein determining one or more focal training data cases from among the set of training data cases based at least in part on the one or more conditions comprises:
determining one or more focal training data cases from among the set of training data cases based at least in part on identifier contribution allocation.
3. The method of claim 2 , further comprising determining the identifier contribution allocation comprises based at least in part on a function of an aggregate identifier contribution allocation for each value of an associated identifier and a number of occurrences of each value of the identifier.
4. The method of claim 3 , further comprising determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on setting an identical aggregate identifier contribution allocation for each value of the identifier.
5. The method of claim 3 , further comprising determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on setting a random aggregate identifier contribution allocation for each value of the identifier.
6. The method of claim 3 , further comprising determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on a function of a total number of cases for each value of the identifier and a total number of cases for the identifier.
7. The method of claim 3 , further comprising determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on setting a received aggregate identifier contribution allocation for each value of the identifier.
8. The method of claim 1 , wherein determining one or more focal training data cases from among the set of training data cases based at least in part on the one or more conditions comprises:
determining one or more focal training data cases from among the set of training data cases based at least in part on two or more identifier contribution allocations.
9. The method of claim 1 , wherein determining one or more focal training data cases from among the set of training data cases based at least in part on the value for the first undetermined feature and any previously-determined values for subsequent undetermined features comprises:
determining the one or more focal training data cases from among the set of training data cases based at least in part on the value for the first undetermined feature and any previously-determined values for subsequent undetermined features and the one or more conditions.
10. A system configured to perform a machine-executed operation by executing instructions, wherein said instructions are instructions which, when executed by one or more computing devices, cause performance of a method comprising:
receiving a request for generation of synthetic data based on a set of training data cases;
for each synthetic data case in the synthetic data,
determining a first undetermined feature in the synthetic data case by:
determining one or more focal training data cases from among the set of training data cases based at least in part on one or more conditions;
determining a value for the first undetermined feature in the synthetic data case based at least in part on the one or more focal training data cases;
using the value for the first undetermined feature in the synthetic data case;
determining subsequent undetermined features in the synthetic data case by:
determining one or more focal training data cases from among the set of training data cases based at least in part on the value for the first undetermined feature and any previously-determined values for subsequent undetermined features;
determining a value for the subsequent undetermined feature in the synthetic data case based at least in part on the one or more focal training data cases;
using the value for the subsequent undetermined feature in the synthetic data case;
continuing to determine subsequent undetermined features until there are no more undetermined features in the synthetic data case;
causing control of a controllable system using a computer-based reasoning model that was determined at least in part based on the synthetic data cases in the synthetic data.
11. The system of claim 10 , wherein determining one or more focal training data cases from among the set of training data cases based at least in part on the one or more conditions comprises:
determining one or more focal training data cases from among the set of training data cases based at least in part on identifier contribution allocation.
12. The system of claim 11 , wherein the method further comprises determining the identifier contribution allocation comprises based at least in part on a function of an aggregate identifier contribution allocation for each value of an associated identifier and a number of occurrences of each value of the identifier.
13. The system of claim 12 , wherein the method further comprises determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on setting an identical aggregate identifier contribution allocation for each value of the identifier.
14. The system of claim 12 , wherein the method further comprises determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on setting a random aggregate identifier contribution allocation for each value of the identifier.
15. The system of claim 12 , wherein the method further comprises determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on a function of a total number of cases for each value of the identifier and a total number of cases for the identifier.
16. A non-transitory computer readable medium storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform a method of:
receiving a request for generation of synthetic data based on a set of training data cases;
for each synthetic data case in the synthetic data,
determining at least one first undetermined feature in the synthetic data case by:
determining one or more focal training data cases from among the set of training data cases based at least in part on one or more conditions;
determining a value for the first undetermined feature in the synthetic data case based at least in part on the one or more focal training data cases;
using the value for the first undetermined feature in the synthetic data case;
determining subsequent undetermined features in the synthetic data case by:
determining one or more focal training data cases from among the set of training data cases based at least in part on the value for the first undetermined feature and any previously-determined values for subsequent undetermined features;
determining a value for the subsequent undetermined feature in the synthetic data case based at least in part on the one or more focal training data cases;
using the value for the subsequent undetermined feature in the synthetic data case;
continuing to determine subsequent undetermined features until there are no more undetermined features in the synthetic data case;
causing control of a controllable system using a computer-based reasoning model that was determined at least in part based on the synthetic data cases in the synthetic data.
17. The non-transitory computer readable medium of claim 16 , wherein determining one or more focal training data cases from among the set of training data cases based at least in part on the one or more conditions comprises:
determining one or more focal training data cases from among the set of training data cases based at least in part on identifier contribution allocation.
18. The non-transitory computer readable medium of claim 17 , wherein the method further comprises determining the identifier contribution allocation comprises based at least in part on a function of an aggregate identifier contribution allocation for each value of an associated identifier and a number of occurrences of each value of the identifier.
19. The non-transitory computer readable medium of claim 18 , wherein the method further comprises determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on setting an identical aggregate identifier contribution allocation for each value of the identifier.
20. The non-transitory computer readable medium of claim 18 , wherein the method further comprises determining the aggregate identifier contribution allocation for each value of the identifier based at least in part on setting a random aggregate identifier contribution allocation for each value of the identifier.