Determining model parameters using secret sharing
This disclosure relates to determining model parameters using secret sharing. In some aspects, a first data party device obtains a first share of a Hessian matrix for a data processing model. The first data party device obtains, using secret sharing with the second data party device, a first share of a product of a random number matrix and the Hessian matrix. The first data party device, determines a first share of a first inverse matrix based on a second inverse matrix and the first share of the random number matrix. The first data party device determines the first inverse matrix, a first share of a product of the first inverse matrix and a gradient of a loss function of the data processing model, and a first share of a new model parameter for the data processing model.
1. A computer-implemented model parameter determining method, comprising:
obtaining, by a first data party device, a first share of a Hessian matrix for a data processing model, wherein the Hessian matrix is secretly shared between the first data party device and a second data party device of a cooperation partner;
obtaining, by the first data party device and using secret sharing with the second data party device, a first share of a product of a random number matrix and the Hessian matrix;
providing, to the second data party device, the first share of the product of the random number matrix and the Hessian matrix;
in response to providing the first share of the product of the random number matrix and the Hessian matrix to the second data party device, receiving, by the first data party device, a second inverse matrix from the second data party device, wherein the second inverse matrix is an inverse of the product of the random number matrix and the Hessian matrix;
determining, by the first data party device, a first share of a first inverse matrix based on the second inverse matrix and the first share of the random number matrix, wherein the first inverse matrix is an inverse of the Hessian matrix;
determining, by the first data party device, the first inverse matrix based on the first share of the first inverse matrix and a second share of the first inverse matrix received from the second data party device, wherein the second share of the first inverse matrix is based on the second inverse matrix and the second share of the random number matrix; and
determining a first share of a product of the first inverse matrix and a gradient of a loss function of the data processing model using a first share of the gradient of the loss function;
determining, a first share of a new model parameter for the data processing model based on an original model parameter of the data processing model and the first share of the product of the first inverse matrix and the gradient of the loss function; and
determining the new model parameter based on the first share of the new model parameter and a second share of the new model parameter received from the second data party device.
2. The computer-implemented method of claim 1 , further comprising updating the data processing model with the new model parameter.
3. The computer-implemented method of claim 1 , wherein the data processing model is a machine learning model and the new model parameter comprises one of a weight or a bias of the machine learning model.
4. The computer-implemented method of claim 1 , wherein determining the first share of the new model parameter comprises subtracting the first share of the product of the first inverse matrix and the gradient of the loss function from the original model parameter.
5. The computer-implemented method of claim 1 , wherein the second share of the new model parameter is based on the original model parameter of the data processing model and a second share of the product of the first inverse matrix and the gradient of the loss function.
6. The computer-implemented method of claim 5 , wherein the second share of the product of the first inverse matrix and the gradient of the loss function comprises a product of the first inverse matrix and a second share of the gradient of the loss function secretly shared with the second data party device.
7. The computer-implemented method of claim 1 , wherein determining, by the first data party device, the first share of a first inverse matrix based on the second inverse matrix and the first share of the random number matrix comprises determining a product of the second inverse matrix and the first share of the random number matrix.
8. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:
obtaining, by a first data party device, a first share of a Hessian matrix for a data processing model, wherein the Hessian matrix is secretly shared between the first data party device and a second data party device of a cooperation partner;
obtaining, by the first data party device and using secret sharing with the second data party device, a first share of a product of a random number matrix and the Hessian matrix;
providing, to the second data party device, the first share of the product of the random number matrix and the Hessian matrix;
in response to providing the first share of the product of the random number matrix and the Hessian matrix to the second data party device, receiving, by the first data party device, a second inverse matrix from the second data party device, wherein the second inverse matrix is an inverse of the product of the random number matrix and the Hessian matrix;
determining, by the first data party device, a first share of a first inverse matrix based on the second inverse matrix and the first share of the random number matrix, wherein the first inverse matrix is an inverse of the Hessian matrix;
determining, by the first data party device, the first inverse matrix based on the first share of the first inverse matrix and a second share of the first inverse matrix received from the second data party device, wherein the second share of the first inverse matrix is based on the second inverse matrix and the second share of the random number matrix; and
determining a first share of a product of the first inverse matrix and a gradient of a loss function of the data processing model using a first share of the gradient of the loss function;
determining, a first share of a new model parameter for the data processing model based on an original model parameter of the data processing model and the first share of the product of the first inverse matrix and the gradient of the loss function; and
determining the new model parameter based on the first share of the new model parameter and a second share of the new model parameter received from the second data party device.
9. The non-transitory, computer-readable medium of claim 8 , wherein the operations comprise updating the data processing model with the new model parameter.
10. The non-transitory, computer-readable medium of claim 8 , wherein the data processing model is a machine learning model and the new model parameter comprises one of a weight or a bias of the machine learning model.
11. The non-transitory, computer-readable medium of claim 8 , wherein determining the first share of the new model parameter comprises subtracting the first share of the product of the first inverse matrix and the gradient of the loss function from the original model parameter.
12. The non-transitory, computer-readable medium of claim 8 , wherein the second share of the new model parameter is based on the original model parameter of the data processing model and a second share of the product of the first inverse matrix and the gradient of the loss function.
13. The non-transitory, computer-readable medium of claim 12 , wherein the second share of the product of the first inverse matrix and the gradient of the loss function comprises a product of the first inverse matrix and a second share of the gradient of the loss function secretly shared with the second data party device.
14. A computer-implemented system, comprising:
one or more computers; and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform operations comprising:
obtaining, by a first data party device, a first share of a Hessian matrix for a data processing model, wherein the Hessian matrix is secretly shared between the first data party device and a second data party device of a cooperation partner;
obtaining, by the first data party device and using secret sharing with the second data party device, a first share of a product of a random number matrix and the Hessian matrix;
providing, to the second data party device, the first share of the product of the random number matrix and the Hessian matrix;
in response to providing the first share of the product of the random number matrix and the Hessian matrix to the second data party device, receiving, by the first data party device, a second inverse matrix from the second data party device, wherein the second inverse matrix is an inverse of the product of the random number matrix and the Hessian matrix;
determining, by the first data party device, a first share of a first inverse matrix based on the second inverse matrix and the first share of the random number matrix, wherein the first inverse matrix is an inverse of the Hessian matrix;
determining, by the first data party device, the first inverse matrix based on the first share of the first inverse matrix and a second share of the first inverse matrix received from the second data party device, wherein the second share of the first inverse matrix is based on the second inverse matrix and the second share of the random number matrix; and
determining a first share of a product of the first inverse matrix and a gradient of a loss function of the data processing model using a first share of the gradient of the loss function;
determining, a first share of a new model parameter for the data processing model based on an original model parameter of the data processing model and the first share of the product of the first inverse matrix and the gradient of the loss function; and
determining the new model parameter based on the first share of the new model parameter and a second share of the new model parameter received from the second data party device.
15. The computer-implemented system of claim 14 , wherein the operations comprise updating the data processing model with the new model parameter.
16. The computer-implemented system of claim 14 , wherein the data processing model is a machine learning model and the new model parameter comprises one of a weight or a bias of the machine learning model.
17. The computer-implemented system of claim 14 , wherein determining the first share of the new model parameter comprises subtracting the first share of the product of the first inverse matrix and the gradient of the loss function from the original model parameter.
18. The computer-implemented system of claim 14 , wherein the second share of the new model parameter is based on the original model parameter of the data processing model and a second share of the product of the first inverse matrix and the gradient of the loss function.
19. The computer-implemented system of claim 18 , wherein the second share of the product of the first inverse matrix and the gradient of the loss function comprises a product of the first inverse matrix and a second share of the gradient of the loss function secretly shared with the second data party device.
20. The computer-implemented system of claim 14 , wherein determining, by the first data party device, the first share of a first inverse matrix based on the second inverse matrix and the first share of the random number matrix comprises determining a product of the second inverse matrix and the first share of the random number matrix.