Machine learning training with enforced differential privacy using secure multi-party computation
Methods and systems for training machine learning models with enforced differential privacy using Secure Multi-Party Computation (SMPC) protocols are disclosed. A method for generating random bits for a Secure Multi-Party Computation (SMPC) protocol in a machine learning model training system includes locally generating, at each participating node, a plurality of random bits using a cryptographically secure pseudorandom number generator (CSPRNG). The locally generated random bits provided as input to the SMPC protocol and combined using an exclusive or (XOR) or addition operation to produce a set of combined random bits. A statistical sampling process for Differential Privacy (DP) is performed using the combined random bits as inputs. The perturbed training data is then utilized in the SMPC protocol to train the machine learning model.
1 . A system for enhancing privacy and security of machine learning model training, the system comprising:
a processor configured to:
execute a Secure Multi-Party Computation (SMPC) protocol on input data to train a machine learning model, wherein the SMPC protocol includes a Differential Privacy (DP) technique applied to an output of the SMPC protocol to ensure that the input data for the machine learning model remains private by limiting potential exposure of individual data contributions during the training of the machine learning model;
encrypt the input data before performing deterministic computations to ensure data security during the execution of the SMPC protocol in the training of the machine learning model;
securely combine locally generated random bits into combined random bits for each different party using an exclusive OR (XOR) operation within the SMPC protocol to ensure unpredictability and uniform distribution of the combined random bits;
process the combined random bits using a cryptographic hash function to enhance randomness and security of the combined random bits; and
aggregate results of the deterministic computations performed by different parties involved in the SMPC protocol to produce a collective output used in the training of the machine learning model.
2 . The system of claim 1 , wherein the processor is configured to initialize secure channels between participating parties using encryption protocols that ensure end-to-end data confidentiality and integrity.
3 . The system of claim 1 , wherein the Differential Privacy (DP) technique further comprises randomly perturbing data from individual training records during the training of the machine learning model.
4 . The system of claim 3 , further comprising randomly perturbing data from intermediate training records alone or in combination with the individual training records.
5 . The system of claim 1 , wherein the SMPC protocol supports secure data exchange among multiple parties without revealing individual inputs.
6 . The system of claim 1 , further comprising a secure computation environment to support the execution of the SMPC protocol and Differential Privacy (DP) techniques.
7 . A method for generating random bits for Secure Multi-Party Computation (SMPC) protocol in a machine learning model training system, the method comprising:
locally generating a plurality of random bits at each participating node using a cryptographically secure pseudorandom number generator (CSPRNG);
securely combining the locally generated random bits into combined random bits at each node using an exclusive OR (XOR) operation within the SMPC protocol to ensure unpredictability and uniform distribution of the combined random bits;
utilizing the combined random bits as input data for a differential privacy (DP) perturbation function applied to training data of a machine learning model to create altered training data; and
incorporating the altered training data directly into the SMPC protocol for training the machine learning model, securely.
8 . The method of claim 7 , wherein the SMPC protocol supports secure data exchange among multiple parties without revealing individual inputs.
9 . The method of claim 7 , further comprising encrypting the input data before performing computations to ensure data security during execution of the SMPC protocol in the training of the machine learning model.
10 . The method of claim 7 , further comprising randomly perturbing data from individual training records during a statistical sampling process.
11 . The method of claim 7 , further comprising randomly perturbing data from intermediate training records during a statistical sampling process.
12 . The method of claim 7 , further comprising initializing secure channels between participating nodes using encryption protocols to ensure end-to-end data confidentiality and integrity.
13 . The method of claim 7 , wherein the combined random bits are further processed using a cryptographic hash function to enhance randomness and security of the combined random bits.
14 . A system for preventing model inversion attacks during training of a machine learning model, the system comprising: a processor configured to execute a Secure Multi-Party Computation (SMPC) protocol; wherein the SMPC protocol employs Differential Privacy (DP) techniques that include generating pseudo-random samples drawn from a statistical distribution; and wherein the processor is further configured to:
receive, from each of a plurality of participating parties, locally generated random bits generated independently using a cryptographically secure pseudorandom number generator (CSPRNG);
combine the locally generated random bits from the plurality of participating parties using an exclusive OR (XOR) or addition operation to produce combined random bits;
utilize the combined random bits as inputs to a statistical sampling process for the Differential Privacy (DP) techniques; and
enforce that the pseudo-random samples used in the Differential Privacy (DP) techniques are derived from the combined random bits such that manipulation of randomness by any single participating party is prevented.
15 . The system of claim 14 , wherein the processor is configured to initialize secure channels between the plurality of participating parties using encryption protocols that ensure end-to-end data confidentiality and integrity.
16 . The system of claim 14 , wherein the Differential Privacy (DP) techniques include randomly perturbing data from individual training records during training of a machine learning model.
17 . The system of claim 14 , wherein the Differential Privacy (DP) techniques include randomly perturbing data from intermediate training records during training of a machine learning model.
18 . The system of claim 17 , wherein the SMPC protocol supports secure data exchange among multiple parties without revealing individual inputs.
19 . The system of claim 18 , wherein the processor is configured to initialize the Secure Multi-Party Computation (SMPC) protocol, incorporate the Differential Privacy (DP) techniques into the SMPC protocol, ensure true randomness of the generated pseudo-random samples, execute the SMPC protocol with DP-enhanced training data as inputs, and output a trained machine learning model resistant to model inversion attacks.