Efficient verification of machine learning applications
An example operation may include one or more of generating, by a training participant client comprising a training dataset, a plurality of transaction proposals that each correspond to a training iteration for machine learning model training related to stochastic gradient descent, the machine learning model training comprising a plurality of training iterations, the transaction proposals comprising a gradient calculation performed by the training participant client, a batch from the private dataset, a loss function, and an original model parameter, receiving, by one or more endorser nodes of peers of a blockchain network, the plurality of transaction proposals, and evaluating each transaction proposal.
1. A system, comprising:
a training participant client in a blockchain network, the training participant client comprising a processor that when executing one or more instructions stored in a memory configures the training participant client to:
generate, using a dataset, a gradient calculation corresponding to a training iteration for machine learning model training comprising a plurality of training iterations, and
generate a plurality of transaction proposals that each correspond to a training iteration of the plurality of training iterations, each transaction proposal comprising a corresponding gradient calculation, a batch comprising a subset of samples from the dataset, a loss function identifying a cost metric, and an original model parameter wherein the corresponding gradient calculation is stored in a blockchain of the blockchain network; and
an endorser node in the blockchain network, the endorser node comprising a processor that when executing one or more instructions stored in a memory configures endorser node to:
receive the plurality of transaction proposals from the training participant client,
calculate a first difference between the loss function at a first point of the two different points and the loss function at a second point of the two different points,
calculate a second difference between the first difference and a product of a size of the batch, a random scalar, and a dot product between the gradient calculation and a random direction,
validate the corresponding gradient calculation based on the second difference being less than a predetermined threshold parameter,
endorse a transaction proposal in response to the validation, and
send the endorsed transaction proposal to the training participation client.
2. The system of claim 1 , wherein the predetermined threshold parameter is application dependent.
3. The system of claim 1 , wherein the endorser node is further configured to:
generate the random scalar and the random direction; and
obtain a new model parameter by perturbing the original model parameter in the random direction with a step size equal to a value of the random scalar.
4. The system of claim 3 , wherein, when the endorser node calculates the first difference, the endorser node is further configured to:
evaluate a difference in the loss function of all the samples in the batch between the new model parameter and the original model parameter.
5. The system of claim 1 , wherein, in response to a determination that the second difference is not less than the predetermined threshold parameter, the endorser node is configured to:
reject the corresponding transaction proposal.
6. The system of claim 1 , wherein, when the endorser node validates the corresponding gradient calculation, the endorser node is further configured to: is further configured to:
verify a correctness of the corresponding gradient calculation based on parameters obtained from the corresponding transaction proposal.
7. A method, comprising:
generating, by a training participant client using a dataset, a gradient calculation corresponding to a training iteration for machine learning model training comprising a plurality of training iterations, wherein the training participant client is in a blockchain network;
generating, by the training participant client, a plurality of transaction proposals that each correspond to a training iteration of the plurality of training iterations, each transaction proposal comprising a corresponding gradient calculation, a batch comprising a subset of samples from the dataset, a loss function identifying a cost metric, and an original model parameter wherein the corresponding gradient calculation is stored in a blockchain of the blockchain network;
receiving, by an endorser node in the blockchain network, the plurality of transaction proposals from the training participant client,
calculating a first difference between the loss function at a first point of the two different points and the loss function at a second point of the two different points;
calculating a second difference between the first difference and a product of a size of the batch, a random scalar, and a dot product between the gradient calculation and a random direction;
validating, by the endorser node, the corresponding gradient calculation based on the second difference being less than a predetermined threshold parameter;
endorsing, by the endorser node, a transaction proposal in response to the validation; and
sending, by the endorser node, the endorsed transaction proposal to the training participation client.
8. The method of claim 7 , wherein the predetermined threshold parameter is application dependent.
9. The method of claim 7 , further comprising
generating the random scalar and the random direction; and
obtaining a new model parameter by perturbing the original model parameter in the random direction with a step size equal to a value of the random scalar.
10. The method of claim 9 , wherein the calculating the first difference further comprises:
evaluating a difference in the loss function of all the samples in the batch between the new model parameter and the original model parameter.
11. The method of claim 7 , wherein, in response to determining that the second difference comparison is not less than the predetermined threshold parameter, the method further comprising:
rejecting the corresponding transaction proposal.
12. The method of claim 7 , wherein the validating the corresponding gradient calculation further comprises:
verifying a correctness of the corresponding gradient calculation based on parameters obtained from the corresponding transaction proposal.
13. A non-transitory computer readable medium comprising one or more instructions that when executed by one or more processors associated with a training participant client of a blockchain network and an endorser node of the blockchain network cause the one or more processors to perform:
generating, by the training participant client using a dataset, a gradient calculation corresponding to a training iteration for machine learning model training comprising a plurality of training iterations, wherein the training participant client is in a blockchain network;
generating, by the training participant client, a plurality of transaction proposals that each correspond to a training iteration of the plurality of training iterations, each transaction proposal comprising a corresponding gradient calculation, a batch comprising a subset of samples from the dataset, a loss function identifying a cost metric, and an original model parameter and wherein the corresponding gradient calculation is stored in a blockchain of the blockchain network;
receiving, by an endorser node in the blockchain network, the plurality of transaction proposals from the training participant client,
calculating a first difference between the loss function at a first point of the two different points and the loss function at a second point of the two different points;
calculating a second difference between the first difference and a product of a size of the batch, a random scalar, and a dot product between the gradient calculation and a random direction;
validating, by the endorser node, the corresponding gradient calculation based on the second difference being less than a predetermined threshold parameter;
endorsing, by the endorser node, a transaction proposal in response to the validation; and
sending, by the endorser node, the endorsed transaction proposal to the training participation client.
14. The non-transitory computer readable medium of claim 13 , wherein the predetermined threshold parameter is application dependent.
15. The non-transitory computer readable medium of claim 13 , wherein the one or more instructions further cause the processor to perform:
generating the random scalar and the random direction; and
obtaining a new model parameter by perturbing the original model parameter in the random direction with a step size equal to a value of the random scalar.
16. The non-transitory computer readable medium of claim 13 , wherein, in response to determining that the second difference comparison is not less than the predetermined threshold parameter, the one or more instructions further cause the processor to perform:
rejecting the corresponding transaction proposal.
17. The non-transitory computer readable medium of claim 13 , wherein the validating the corresponding gradient calculation further comprises:
verifying a correctness of the corresponding gradient calculation based on parameters obtained from the corresponding transaction proposal.
18. The non-transitory computer readable medium of claim 13 , wherein the calculating the first difference further comprises:
evaluating a difference in the loss function of all the samples in the batch between the new model parameter and the original model parameter.