Generating user-based training data for machine learning
In an example embodiment, a machine learning-based solution for generating synthetic CVs that preserve the statistical properties of the original corpus is provided, while providing strong privacy guarantees. As synthetic data do not refer to any natural person and can be generated from anonymized data, they are not subject to data protection regulations.
1 . A system comprising:
at least one hardware processor; and
a computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
accessing a corpus of documents having unstructured data;
extracting candidate attributes from the corpus using named entity recognition and relation extraction;
forming the extracted candidate attributes into a structured dataset;
building a Bayesian network graph, the Bayesian network graph being a directed acyclic graph having the extracted candidate attributes as nodes with edges between nodes representing relationships between candidate attributes, wherein each node includes a value and is associated with a function comprising a conditional probability distribution;
adding Laplace noise to the conditional probability distribution of a plurality of nodes in the Bayesian network graph to provide differential privacy;
sampling new values for a plurality of nodes in the Bayesian network graph; and
using the sampled new values as training data for a machine learning algorithm to train a machine learning model.
2 . The system of claim 1 , wherein the operations further comprise generating artificial personal details for the sampled new values; and
wherein the using the sampled new values includes using the sampled new values along with the generated personal details as training data for the machine learning algorithm to train the machine learning model.
3 . The system of claim 1 , wherein the operations further comprise:
removing bias from the sampled new values by adjusting the conditional probability distributions.
4 . The system of claim 1 , wherein each function takes as input a set of possible values for a corresponding node's parent variables and gives as output a probability distribution on the corresponding node's values.
5 . The system of claim 1 , wherein the sampling includes following conditional dependencies of each node in the Bayesian network graph, maintaining consistency and properties of an original corpus of documents.
6 . The system of claim 2 , wherein the operations further comprise:
combining the sampled new values with the generated personal details using an auto-regressive pre-trained natural language generation (NLG) model.
7 . The system of claim 1 , wherein the extracting further comprises using a natural language processing (NLP) machine learning model.
8 . A method comprising:
accessing a corpus of documents having unstructured data;
extracting candidate attributes from the corpus using named entity recognition and relation extraction;
forming the extracted candidate attributes into a structured dataset;
building a Bayesian network graph, the Bayesian network graph being a directed acyclic graph having the extracted candidate attributes as nodes with edges between nodes representing relationships between candidate attributes, wherein each node includes a value and is associated with a function comprising a conditional probability distribution;
adding Laplace noise to the conditional probability distribution of a plurality of nodes in the Bayesian network graph to provide differential privacy;
sampling new values for a plurality of nodes in the Bayesian network graph; and
using the sampled new values as training data for a machine learning algorithm to train a machine learning model.
9 . The method of claim 8 , further comprising generating artificial personal details for the sampled new values; and
wherein the using the sampled new values includes using the sampled new values along with the generated personal details as training data for the machine learning algorithm to train the machine learning model.
10 . The method of claim 8 , further comprising:
removing bias from the sampled new values by adjusting the conditional probability distributions.
11 . The method of claim 8 , wherein each function takes as input a set of possible values for a corresponding node's parent variables and gives as output a probability distribution on the corresponding node's values.
12 . The method of claim 8 , wherein the sampling includes following conditional dependencies of each node in the Bayesian network graph, maintaining consistency and properties of an original corpus of documents.
13 . The method of claim 9 , further comprising:
combining the sampled new values with the generated personal details using an auto-regressive pre-trained natural language generation (NLG) model.
14 . The method of claim 8 , wherein the extracting further comprises using a natural language processing (NLP) machine learning model.
15 . A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
accessing a corpus of documents having unstructured data;
extracting candidate attributes from the corpus using named entity recognition and relation extraction;
forming the extracted candidate attributes into a structured dataset;
building a Bayesian network graph, the Bayesian network graph being a directed acyclic graph having the extracted candidate attributes as nodes with edges between nodes representing relationships between candidate attributes, wherein each node includes a value and is associated with a function comprising a conditional probability distribution;
adding Laplace noise to the conditional probability distribution of a plurality of nodes in the Bayesian network graph to provide differential privacy;
sampling new values for a plurality of nodes in the Bayesian network graph; and
using the sampled new values as training data for a machine learning algorithm to train a machine learning model.
16 . The non-transitory machine-readable medium of claim 15 , further comprising generating artificial personal details for the sampled new values; and
wherein the using the sampled new values includes using the sampled new values along with the generated personal details as training data for the machine learning algorithm to train the machine learning model.
17 . The non-transitory machine-readable medium of claim 15 , further comprising:
removing bias from the sampled new values by adjusting the conditional probability distributions.
18 . The non-transitory machine-readable medium of claim 15 , wherein each function takes as input a set of possible values for a corresponding node's parent variables and gives as output a probability distribution on the corresponding node's values.
19 . The non-transitory machine-readable medium of claim 15 , wherein the sampling includes following conditional dependencies of each node in the Bayesian network graph, maintaining consistency and properties of an original corpus of documents.
20 . The non-transitory machine-readable medium of claim 16 , further comprising:
combining the sampled new values with the generated personal details using an auto-regressive pre-trained natural language generation (NLG) model.