Adversarial learning of privacy preserving representations

1 . A system, comprising:

at least one processor; and

at least one memory including program code which when executed by the at least one processor provides operations comprising:

training a first machine learning model to generate a first representation of a first user data having a plurality of private attributes and a plurality of non-private attributes, the training comprising partitioning the plurality of private attributes into similar private attributes and dissimilar private attributes;

training, based at least on the first representation of a target portion of the first user data and a residual portion of the first user data generated by an encoder, a non-private attribute predictor and a private attribute predictor to maintain a separation between the plurality of private attributes and the plurality of non-private attributes;

training, based at least on the first representation of the target portion of the first user data and a residual portion of the first user data generated by the encoder, an adversarial private attribute predictor and an adversarial non-private attribute predictor to maximize an entropy amongst the similar private attributes of the first user data;

optimizing, based at least on outputs of the private attribute predictor, the non-private attribute predictor, the adversarial private attribute predictor, and the adversarial non-private attribute predictor, a performance of the encoder generating the first representation of the first user data during the training of the first machine learning model;

applying the trained first machine learning model to generate a second representation of a second user data having the plurality of private attributes and the plurality of non-private attributes; and

performing one or more downstream tasks by at least training a second machine learning model using the second representation of the second user data without revealing the plurality of private attributes associated with the second user data,

wherein the training using the second representation of the second user data enables the second machine learning model to generate a classification based on the plurality of non-private attributes but prevents the second machine learning model from generating the classification based on the plurality of private attributes,

wherein the similar private attributes include a set of labels corresponding to a number k of largest scores given an observational input, and

wherein the dissimilar private attributes include a complement to the number k of largest scores.

2 . The system of claim 1 , wherein the training of the first machine learning model includes shifting the entropy amongst the plurality of private attributes to maximize the entropy amongst the similar private attributes.

3 . The system of claim 1 , wherein the first machine learning model is further trained to generate the first representation to preserve a first information associated with the plurality of non-private attributes and omit a second information associated with the plurality of private attributes.

4 . The system of claim 1 , wherein the first machine learning model includes the encoder trained to generate the first representation to include the target portion including the plurality of non-private attributes and the residual portion including the plurality of private attributes.

5 . The system of claim 4 , wherein the first user data is classified based on the target portion of the first representation but not the residual portion of the first representation.

6 . The system of claim 4 , wherein the first machine learning model includes the non-private attribute predictor, the private attribute predictor, the adversarial private attribute predictor, and the adversarial non-private attribute predictor.

7 . The system of claim 6 , wherein the training of the first machine learning model includes subjecting the non-private attribute predictor, the private attribute predictor, the adversarial private attribute predictor, and the adversarial non-private attribute predictor to an adversarial training.

8 . The system of claim 1 , wherein the plurality of non-private attributes include one or more coarse labels associated with the first user data, and wherein the plurality of private attributes include one or more fine labels associated with the first user data.

9 . The system of claim 1 , wherein the similar private attributes include attributes that are common to user data sharing a same label and/or a same classification.

10 . A computer-implemented method, comprising:

training a first machine learning model to generate a first representation of a first user data having a plurality of private attributes and a plurality of non-private attributes, the training comprising partitioning the plurality of private attributes into similar private attributes and dissimilar private attributes;

training, based at least on the first representation of a target portion of the first user data and a residual portion of the first user data generated by an encoder, a non-private attribute predictor and a private attribute predictor to maintain a separation between the plurality of private attributes and the plurality of non-private attributes;

training, based at least on the first representation of the target portion of the first user data and a residual portion of the first user data generated by the encoder, an adversarial private attribute predictor and an adversarial non-private attribute predictor to maximize an entropy amongst the similar private attributes of the first user data;

optimizing, based at least on outputs of the private attribute predictor, the non-private attribute predictor, the adversarial private attribute predictor, and the adversarial non-private attribute predictor, a performance of the encoder generating the first representation of the first user data during the training of the first machine learning model;

applying the trained first machine learning model to generate a second representation of a second user data having the plurality of private attributes and the plurality of non-private attributes; and

performing one or more downstream tasks by at least training a second machine learning model using the second representation of the second user data without revealing the plurality of private attributes associated with the second user data,

wherein the training using the second representation of the second user data enables the second machine learning model to generate a classification based on the plurality of non-private attributes but prevents the second machine learning model from generating the classification based on the plurality of private attributes,

wherein the similar private attributes include a set of labels corresponding to a number k of largest scores given an observational input, and

wherein the dissimilar private attributes include a complement to the number k of largest scores.

11 . The method of claim 10 , wherein the training of the first machine learning model includes shifting the entropy amongst the plurality of private attributes to maximize the entropy amongst the similar private attributes.

12 . The method of claim 10 , wherein the first machine learning model is further trained to generate the first representation to preserve a first information associated with the plurality of non-private attributes and omit a second information associated with the plurality of private attributes.

13 . The method of claim 10 , wherein the first machine learning model includes the encoder trained to generate the first representation to include the target portion including the plurality of non-private attributes and the residual portion including the plurality of private attributes.

14 . The method of claim 13 , wherein the first user data can be classified based on the target portion of the first representation but not the residual portion of the first representation.

15 . The method of claim 13 , wherein the first machine learning model includes the non-private attribute predictor, the private attribute predictor, the adversarial private attribute predictor, and the adversarial non-private attribute predictor, and wherein the training of the first machine learning model includes subjecting the non-private attribute predictor, the private attribute predictor, the adversarial private attribute predictor, and the adversarial non-private attribute predictor to an adversarial training.

16 . The method of claim 10 , wherein the plurality of non-private attributes include one or more coarse labels associated with the first user data, and wherein the plurality of private attributes include one or more fine labels associated with the first user data.

17 . The method of claim 10 , wherein the similar private attributes include attributes that are common to user data sharing a same label and/or a same classification.

18 . A non-transitory computer readable medium storing instructions, which when executed by at least one data processor, result in operations comprising:

training a first machine learning model to generate a first representation of a first user data having a plurality of private attributes and a plurality of non-private attributes, the training comprising partitioning the plurality of private attributes into similar private attributes and dissimilar private attributes;

training, based at least on the first representation of a target portion of the first user data and a residual portion of the first user data generated by an encoder, a non-private attribute predictor and a private attribute predictor to maintain a separation between the plurality of private attributes and the plurality of non-private attributes;

training, based at least on the first representation of the target portion of the first user data and a residual portion of the first user data generated by the encoder, an adversarial private attribute predictor and an adversarial non-private attribute predictor to maximize an entropy amongst the similar private attributes of the first user data;

optimizing, based at least on outputs of the private attribute predictor, the non-private attribute predictor, the adversarial private attribute predictor, and the adversarial non-private attribute predictor, a performance of the encoder generating the first representation of the first user data during the training of the first machine learning model;

applying the trained first machine learning model to generate a second representation of a second user data having the plurality of private attributes and the plurality of non-private attributes; and

performing one or more downstream tasks by at least training a second machine learning model using the second representation of the second user data without revealing the plurality of private attributes associated with the second user data,

wherein the training using the second representation of the second user data enables the second machine learning model to generate a classification based on the plurality of non-private attributes but prevents the second machine learning model from generating the classification based on the plurality of private attributes,

wherein the similar private attributes include a set of labels corresponding to a number k of largest scores given an observational input, and

wherein the dissimilar private attributes include a complement to the number k of largest scores.