SELF-SUPERVISED DATA OBFUSCATION IN FOUNDATION MODELS
Provided are methods and system for obtaining, by a computer system, a machine learning/machine learning model; obtaining, by the computer system, a training data set; training, with the computer system, an obfuscation transform based on the machine learning/machine learning model and the training data set; and storing, with the computer system, the obfuscation transform in memory.
1 . A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations comprising:
obtaining, by a computer system, a dataset;
training, with the computer system, one or more machine learning models as an autoencoder to generate as output a reconstruction of a record in the dataset based on an input of the record in the dataset, wherein the autoencoder comprises a deterministic layer and wherein training is based on optimization of a value indicative of reconstruction loss;
adding, with the computer system, one or more stochastic noise layers to the trained one or more machine learning models of the autoencoder;
adjusting, with the computer system, parameters of the stochastic noise layers according to an objective function that is differentiable; and
storing, with the computer system, the one or more machine learning models of the autoencoder with the stochastic noise layers in memory.
2 . A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations comprising:
obtaining, by a computer system, a machine learning model;
obtaining, by the computer system, a training data set;
training, by the computer system, an obfuscation transform based on the machine learning model and the training data set by self-supervision; and
storing, with the computer system, the trained obfuscation transform in memory.
3 . The medium of claim 2 , wherein the machine learning model is a generative artificial intelligence (AI) model trained with self-supervision, and the trained obfuscation transform is configured to transform records into obfuscated records that are correctly processed by the machine learning model despite the obfuscation.
4 . The medium of claim 2 , wherein the machine learning model is a foundation model, where the foundation model is operative to perform a plurality of tasks at inference time with capabilities that emerged during training and were not explicitly measured by an objective function used to train the foundation model.
5 . The medium of claim 2 , wherein training the obfuscation transform comprises:
adding an obfuscation transform to at least one of the training data set and the machine learning model; and
adjusting parameters of the obfuscation transform according to an objective function that is differentiable.
6 . The medium of claim 2 , wherein the obfuscation transform comprises a stochastic noise layer and wherein training the obfuscation transform comprises determining parameters of distribution of stochastic noise of the stochastic noise layer.
7 . The medium of claim 6 , wherein the stochastic noise layer is applied to input into the machine learning model.
8 . The medium of claim 6 , wherein the stochastic noise layer is applied to input into a layer of the machine learning model.
9 . The medium of claim 8 , wherein the stochastic noise layer is applied to embedded values within the machine learning model.
10 . The medium of claim 6 , wherein the trained obfuscation transform is configured to obfuscate data designated as being sensitive.
11 . The medium of claim 2 , wherein
the machine learning model is an ensemble model;
the machine learning model comprises an image-based model, language-based model, or tabular-data-based model;
the machine learning model is at least one of an inference model, a classification model, a prediction model, or a transformer;
the obfuscation transform is applied to at least a portion of the ensemble model; and
the obfuscation transform is trained by optimization of an objective function, the objective function minimizing mutual information and minimizing data loss.
12 . The medium of claim 2 , further comprising tuning the machine learning model based on the training data set.
13 . The medium of claim 12 , further comprising deploying the tuned machine learning model.
14 . The medium of claim 2 , further comprising applying the stored obfuscation transform to a set of production data.
15 . The medium of claim 14 , wherein the stored obfuscation transform is applied to the set of production data to generate obfuscated data and wherein the obfuscated data is input into the machine learning model.
16 . The medium of claim 15 , wherein the stored obfuscation transform is applied to the set of production data before the set of production data is transmitted to the machine learning model.
17 . The medium of claim 15 , wherein the stored obfuscation transform is applied to the set of production data after the production data is transmitted to the machine learning model.
18 . The medium of claim 2 , further comprising steps for deploying the obfuscation transform to a production dataset.
19 . The medium of claim 2 , further comprising steps for obfuscating a data set based on the obfuscation transform.
20 . A method comprising:
obtaining, with a computer system, a machine learning model;
obtaining, with the computer system, a training data set;
training, with the computer system, an obfuscation transform based on the machine learning model and the training data set; and
storing, with the computer system, the obfuscation transform in memory.