Private artificial neural networks with trusted execution environments and quadratic homomorphic encryption
A method training an artificial neural network (ANN) on a remote host computes, using a trusted process deployed in a trusted execution environment (TEE) on the remote host, a key-pair for a homomorphic encryption scheme and shares, by the trusted process, the public key (PK) of the key-pair with an untrusted process deployed on the remote host. The method splits the training procedure of the ANN between the untrusted process and the trusted process, wherein the untrusted process computes encrypted inputs to neurons of the ANN by means of the homomorphic encryption scheme, while the trusted process computes outputs of the neurons based on the respective encrypted inputs to the neurons as provided by the untrusted process.
1. A computer-implemented method of training an artificial neural network (ANN) on a remote host, the method comprising:
computing, by a trusted process deployed in a trusted execution environment (TEE) on the remote host, a key-pair for a homomorphic encryption scheme and sharing, by the trusted process, the public key (PK) of the key-pair with an untrusted process deployed on the remote host; and
splitting the training procedure of the ANN between the untrusted process and the trusted process, wherein the untrusted process computes encrypted inputs to neurons of the ANN by means of the homomorphic encryption scheme, while the trusted process computes outputs of the neurons based on the respective encrypted inputs to the neurons as provided by the untrusted process.
2. The method according to claim 1 , wherein the homomorphic encryption scheme is parametrized to compute quadratic functions.
3. The method according to claim 1 , further comprising:
encrypting, by the trusted process, the inputs and the parameters, including the weights, of the ANN with the public key (PK);
sending the encrypted inputs and parameters to the untrusted process; and
computing, by the trusted process cooperating with the untrusted process, an output of the ANN and a gradient of the weights, given the encrypted inputs and the encrypted weights.
4. The method according to claim 1 , further comprising an initialization phase, wherein the initialization phrase comprises:
computing, by the trusted process, random weight matrices for all layers of the ANN;
encrypting the random weight matrices with the public key (PK); and
sending the encrypted weight matrices to the untrusted process.
5. The method according to claim 1 , further comprising a feed-forwarding phase, the feed-forward phrase comprising:
sending, by the trusted process for each layer of the ANN, the output of the neurons of a respective layer, encrypted with the public key (PK), to the untrusted process; and
computing, by the untrusted process, an input for the respective subsequent layer of the ANN by executing homomorphic matrix multiplication of the respective encrypted weight matrix and the encrypted output as received from the trusted process.
6. The method according to claim 5 , further comprising:
decrypting, by the trusted process, the input for the respective subsequent layer of the ANN as received from the untrusted process; and
computing, by the trusted process, an output of the respective subsequent layer by computing on the decrypted input the respective activation function.
7. The method according to claim 5 , further comprising:
iterating the feed-forwarding phase, the back-propagation phase and the weights-updating phase over each sample of an ANN training data set.
8. The method according to claim 1 , further comprising a back-propagation phase for minimizing the cost function of the ANN by adjusting the weights, the back-propagation phase comprising:
computing, by the trusted process for each layer of the ANN, a gradient of the weights; and
encrypting the gradients with the public key (PK) and sending the encrypted gradients to the untrusted process.
9. The method according to claim 8 , wherein, at the end of the propagation phase, the untrusted process holds, for each layer of the ANN, a gradient weight matrix, encrypted with the public key (PK).
10. The method according to claim 9 , further comprising a weights-updating phase, the weights-updating phase comprising:
computing, by the untrusted process, based on the encrypted weight matrices and gradient weight matrices, updated weight matrices by executing, for each layer of the ANN, homomorphic matrix addition of the respective encrypted weight matrix and the respective encrypted gradient weight matrix.
11. The method according to claim 1 , further comprising a weight refreshing procedure, the weight refreshing procedure comprising:
sending, by the untrusted process, encrypted weight matrices of each layer of the ANN to the trusted process and discarding them afterwards;
decrypting, by the trusted process, the received encrypted weight matrices to obtain the plaintext weight matrices and encrypting each plaintext weight matrix in a fresh ciphertext; and
sending, by the trusted process, the refreshed encrypted weight matrices to the untrusted process.
12. The method according to claim 11 , wherein the weight refreshing procedure is executed in case a number of weight updates reaches an upper bound as defined by parameters of the homomorphic encryption scheme.
13. The method according to claim 1 , wherein the ANN training data are provided, encrypted under the public key (PK) of the trusted process, by a plurality of different data owners.
14. A host processing system for remote training of an artificial neural network, the host processing system comprising a trusted execution environment (TEE) and an untrusted processing system (UPS) and being configured to:
run a trusted process deployed in the TEE that computes a key-pair for a homomorphic encryption scheme and share the public key (PK) of the key-pair with an untrusted process running on the UPS; and
split the training procedure of the ANN between the untrusted process and the trusted process, wherein the untrusted process is configured to compute encrypted inputs to the neurons of the ANN by means of the homomorphic encryption scheme, while the trusted process is configured to compute outputs of the neurons based on the respective encrypted inputs to the neurons as provided by the untrusted process.
15. A non-transitory computer readable medium for remote training of an artificial neural network with a host processing system comprising a trusted execution environment (TEE), and an untrusted processing system (UPS) the medium comprising program code for configuring the host processing system to:
run a trusted process deployed in the TEE that computes a key-pair for a homomorphic encryption scheme and share the public key (PK) of the key-pair with an untrusted process running on the UPS; and
split the training procedure of the ANN between the untrusted process and the trusted process, wherein the untrusted process is configured to compute encrypted inputs to neurons of the ANN by means of the homomorphic encryption scheme, while the trusted process is configured to compute outputs of the neurons based on the respective encrypted inputs to the neurons as provided by the untrusted process.