Systems and methods for synthetic interventions
A computer-implemented method includes: identifying, from first and second data, interventions common to a target unit and one or more of a plurality of donor units as filtered donor units, the first data corresponding to the target unit under one or more interventions, the second data corresponding to the plurality of donor units each under one or more interventions; identifying, from the first data, third data corresponding to the target unit under the common interventions; identifying, from the second data, fourth data corresponding to the filtered donor units under the common interventions; identifying, from the second data, fifth data corresponding to the filtered donor units under a subject intervention; generating, from the third and fourth data, a learned model representing to a relationship between the target unit and the filtered donor units; applying the learned model to the fifth data to generate the synthetic data; and outputting the synthetic data.
1. A method implemented on one or more computing devices for generating synthetic data for a target unit had the target unit undergone a subject intervention:
identifying, from first and second data, interventions common to the target unit and one or more of a plurality of donor units as filtered donor units, the first data corresponding to the target unit under one or more interventions, the second data corresponding to the plurality of donor units each under one or more interventions;
identifying, from the first data, third data corresponding to the target unit under the common interventions;
identifying, from the second data, fourth data corresponding to the filtered donor units under the common interventions;
identifying, from the second data, fifth data corresponding to the filtered donor units under the subject intervention;
generating, from the third and fourth data, a learned model representing a relationship between the target unit and the filtered donor units;
applying the learned model to the fifth data to generate the synthetic data; and
using the synthetic data to automate one or more decision-making processes related to which interventions should be applied or targeted to which units.
2. The method of claim 1 , further comprising receiving the first and second data from a database.
3. The method of claim 1 , wherein identifying the interventions common to the target unit and one or more of a plurality of donor units includes filtering the first and second data to identify observations associated with common interventions between the target unit and the plurality of donor units that maximizes the minimum between a number of satisfactory observations and a number of the filtered donor units.
4. The method of claim 3 , wherein the number of satisfactory observations is a number of observations for a particular one of the plurality of donor units at points along a given dimension where observations also exist for the target unit at those same points along the same dimension.
5. The method of claim 4 , wherein the dimension is time.
6. The method of claim 1 , wherein generating the learned model includes performing principal component regression (PCR) between the third and fourth data to generate a learned model that defines the unique minimum-norm linear relationship between the target unit and the filtered donor units.
7. The method of claim 1 , further comprising storing the synthetic data in a database.
8. The method of claim 1 , further comprising transmitting the synthetic data to another computing device.
9. The method of claim 8 , wherein the another computing device is part of a system configured to automate the one or more decision-making processes using the synthetic data.
10. The method of claim 1 further comprising validating data including at least the fifth data corresponding to the filtered donor units under the subject intervention.
11. The method of claim 10 wherein the validating includes:
performing a singular value decomposition of the data and inspecting its spectral profile; and
in response to determining that the data does not exhibit low-dimensional structure, pre-processing the data prior to generating the synthetic data.
12. The method of claim 11 wherein the pre-processing includes applying an autoencoder to identify a new low-dimensional representation of the data.
13. The method of claim 10 wherein the validating includes:
performing a subspace inclusion hypothesis test on the data; and
in response to the subspace inclusion hypothesis test passing, determining whether or not accurate synthetic data can be generated for the target unit under the subject intervention,
wherein the synthetic data is generated in response to determining that accurate synthetic data can be generated.
14. A system for generating synthetic data for a target unit had the target unit undergone a subject intervention, the system comprising:
a database configured to store first data corresponding to the target unit under one or more interventions and second data corresponding to a plurality of donor units each under one or more interventions; and
a computing device comprising instructions which, when executed cause the computing device to execute a process including:
identifying, from first and second data, interventions common to the target unit and one or more of a plurality of donor units as filtered donor units;
identifying, from the first data, third data corresponding to the target unit under the common interventions;
identifying, from the second data, fourth data corresponding to the filtered donor units under the common interventions;
identifying, from the second data, fifth data corresponding to the filtered donor units under the subject intervention;
generating, from the third and fourth data, a learned model representing a relationship between the target unit and the filtered donor units;
applying the learned model to the fifth data to generate the synthetic data; and
transmitting the synthetic data to another computing device configured to automate one or more decision-making processes using the synthetic data.
15. The system of claim 14 , wherein identifying the interventions common to the target unit and one or more of a plurality of donor units includes filtering the first and second data to identify observations associated with common interventions between the target unit and the plurality of donor units that maximizes the minimum between a number of satisfactory observations and a number of the filtered donor units.
16. The system of claim 15 , wherein the number of satisfactory observations is a number of observations for a particular one of the plurality of donor units taken at points along a given dimension where observations also exist for the target unit at those same points along the same dimension.
17. The system of claim 16 , wherein the dimension is time.
18. The system of claim 14 , wherein generating the learned model includes performing principal component regression (PCR) between the third and fourth data to generate a learned model that defines the unique minimum-norm linear relationship between the target unit and the filtered donor units.
19. The system of claim 14 , wherein the another computing device is part of a system configured to automate one or more decision-making processes using the synthetic data.