Generative adversarial network employed for decentralized and confidential AI training

1. A computer implemented method executed on a processor for constructing a trained model for a plurality of edge devices in a network having a federated classifier, a generator model, and a discriminator model, the method comprising:

obtaining edge trained models from the plurality of edge devices, each edge trained model being trained independently with data from private data of each edge;

training the generator model and discriminator model by employing the edge trained models and an unlabeled set of data by employing a generative adversarial training procedure;

generating data samples by the trained generator model;

training the federated classifier with the data samples from the generator model;

merging the edge trained models at a cloud server to construct a single trained model; and

deploying the single trained model back to the plurality of edge devices without the private data of each edge.

2. The method of claim 1 , wherein the edge trained models include different structures and different target classes.

3. The method of claim 1 , wherein the network produces the federated classifier without having explicit access to labeled data.

4. The method of claim 1 , wherein the generative adversarial training procedure between the generator model and the discriminator model involves alternately minimizing a first cost function and a second cost function.

5. The method of claim 4 , wherein the first cost function is

L DF (θ F ,θ D ,θ C F ,θ G )=λ 1 ( D ( F ( G ( z,e l ;θ G );θ F );θ D ),0)+λ 2 Σ l∈X real ( D ( F ( I;θ F );θ D ),1)+λ 3 ( C F ( F ( G ( z,e l ;θ G );θ F );θ C F ), l )

and the second cost function is

L G (θ F ,θ D ,θ G )= [−λ 4 ( D ( F ( G ( z,e l ;θ G );θ F );θ D ),0)+λ 5 Σ n:l∈L n ( C n ( G ( z,e l ;θ G )); l )],

where θ are model parameters of each respective component, λ are hyperparameters used for balancing terms in the cost functions, μ is a mean deviation of the normal distribution used for sampling vector z, σ is a standard deviation of the normal distribution used for sampling vector z, D is the discriminator model, G is the generator model, F is the feature extractor, e l is a one-hot vector for label l, l is the loss function, z is a noise seed function, C F is the federated classifier, C n is an edge trained classifier, and I is a real image sampled from a set of real images.

6. The method of claim 1 , wherein the method is executed in an unsupervised fashion without any exchange of training data between the cloud server and the plurality of edge devices.

7. A non-transitory computer-readable storage medium comprising a computer-readable program executed on a processor in a data processing system for constructing a trained model for a plurality of edge devices in a network having a federated classifier, a generator model, and a discriminator model, wherein the computer-readable program when executed on the processor causes a computer to perform the steps of:

obtaining edge trained models from the plurality of edge devices, each edge trained model being trained independently with data from private data of each edge;

training the generator model and discriminator model by employing the edge trained models and an unlabeled set of data by employing a generative adversarial training procedure;

generating data samples by the trained generator model;

training the federated classifier with the data samples from the generator model;

merging the edge trained models at a cloud server to construct a single trained model; and

deploying the single trained model back to the plurality of edge devices without the private data of each edge.

8. The non-transitory computer-readable storage medium of claim 7 , wherein the edge trained models include different structures and different target classes.

9. The non-transitory computer-readable storage medium of claim 7 , wherein the network produces the federated classifier without having explicit access to labeled data.

10. The non-transitory computer-readable storage medium of claim 7 , wherein the generative adversarial training procedure between the generator model and the discriminator model involves alternately minimizing a first cost function and a second cost function.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the first cost function is

L DF (θ F ,θ D ,θ C F ,θ G )=λ 1 ( D ( F ( G ( z,e l ;θ G );θ F );θ D ),0)+λ 2 Σ l∈X real ( D ( F ( I;θ F );θ D ),1)+λ 3 ( C F ( F ( G ( z,e l ;θ G );θ F );θ C F ), l )

and the second cost function is

L G (θ F ,θ D ,θ G )= [−λ 4 ( D ( F ( G ( z,e l ;θ G );θ F );θ D ),0)+λ 5 Σ n:l∈L n ( C n ( G ( z,e l ;θ G )); l )],

, where θ are model parameters of each respective component, λ are hyperparameters used for balancing terms in the cost functions, μ is a mean deviation of the normal distribution used for sampling vector z, σ is a standard deviation of the normal distribution used for sampling vector z, D is the discriminator model, G is the generator model, F is the feature extractor, e l is a one-hot vector for label l, l is the loss function, z is a noise seed function, C F is the federated classifier, C n is an edge trained classifier, and I is a real image sampled from a set of real images.

12. The non-transitory computer-readable storage medium of claim 7 , wherein a learning method is executed in an unsupervised fashion without any exchange of training data between the cloud server and the plurality of edge devices.

13. A system for constructing a trained model, the system comprising:

a plurality of edge devices in a network having a federated classifier, a generator model, and a discriminator model configured to:

obtain edge trained models from the plurality of edge devices, each edge trained model being trained independently with data from private data of each edge;

train the generator model and discriminator model by employing the edge trained models and an unlabeled set of data by employing a generative adversarial training procedure;

generate data samples by the trained generator model;

train the federated classifier with the data samples from the generator model;

merge the edge trained models at a cloud server to construct a single trained model; and

deploy the single trained model back to the plurality of edge devices without the private data of each edge.

14. The system of claim 13 , wherein the edge trained models include different structures and different target classes.

15. The system of claim 13 , wherein the network produces the federated classifier without having explicit access to labeled data.

16. The system of claim 13 , wherein the generative adversarial training procedure between the generator model and the discriminator model involves alternately minimizing a first cost function and a second cost function.

17. The system of claim 16 , wherein the first cost function is

L DF (θ F ,θ D ,θ C F ,θ G )=λ 1 ( D ( F ( G ( z,e l ;θ G );θ F );θ D ),0)+λ 2 Σ l∈X real ( D ( F ( I;θ F );θ D ),1)+λ 3 ( C F ( F ( G ( z,e l ;θ G );θ F );θ C F ), l )

and the second cost function is

L G (θ F ,θ D ,θ G )= [−λ 4 ( D ( F ( G ( z,e l ;θ G );θ F );θ D ),0)+λ 5 Σ n:l∈L n ( C n ( G ( z,e l ;θ G )); l )],

where θ are model parameters of each respective component, λ are hyperparameters used for balancing terms in the cost functions, μ is a mean deviation of the normal distribution used for sampling vector z, σ is a standard deviation of the normal distribution used for sampling vector z, D is the discriminator model, G is the generator model, F is the feature extractor, e l is a one-hot vector for label l, l is the loss function, z is a noise seed function, C F is the federated classifier, C n is an edge trained classifier, and I is a real image sampled from a set of real images.

18. The system of claim 13 , wherein a learning method is executed in an unsupervised fashion without any exchange of training data between the cloud server and the plurality of edge devices.