Optimizing method of distributed training and master computing apparatus

1 . An optimizing method of distributed training, comprising:

training a local model by using one of a plurality of sample sets and a global parameter of a global model to generate a local parameter of the local model;

determining at least one deviation parameter among the local parameter of a plurality of local models trained by the sample sets, wherein a distribution of the at least one deviation parameter is far from a distribution of other local parameters, and the local parameter of the local models is used to update the global parameter of the global model;

deleting the at least one deviation parameter from the local parameter of the local models trained by the sample sets; and

updating the global parameter of the global model with other local parameters other than the at least one deviation parameter;

wherein the local parameter of each of the local models comprises a local correction parameter, the global parameter comprises a global correction parameter, the global correction parameter is obtained by a weighted operation based on the local correction parameter, and the step of determining the at least one deviation parameter among the local parameter of the local models trained by the sample sets comprises:

determining the at least one deviation parameter according to an operation weight used by the local correction parameter of each of the local models in the weighted operation.

2 . The optimizing method of distributed training according to claim 1 , wherein the step of determining the at least one deviation parameter according to the operation weight used by the local correction parameter of each of the local models in the weighted operation comprises:

comparing the operation weight used by the local correction parameter of each of the local models in the weighted operation with a threshold deviation;

using the local correction parameter of one of the local models as the at least one deviation parameter in response to the operation weight used by the local correction parameter of the one of the local models in the weighted operation being less than the threshold deviation; and

prohibiting the local correction parameter of one of the local models as the at least one deviation parameter in response to the operation weight used by the local correction parameter of the one of the local models in the weighted operation not being less than the threshold deviation.

3 . The optimizing method of distributed training according to claim 2 , further comprising:

determining a probability distribution of the operation weight used by the local correction parameter of the local models in the weighted operation; and

taking a value corresponding to a deviation quantile in the probability distribution as the threshold deviation.

4 . The optimizing method of distributed training according to claim 3 , wherein the probability distribution is a t distribution.

5 . The optimizing method of distributed training according to claim 1 , wherein the local correction parameter is a gradient value determined based on a gradient descent method.

6 . The optimizing method of distributed training according to claim 1 , wherein the local parameter of each of the local models comprises a local correction parameter, the global parameter comprises a global correction parameter, and the step of determining the at least one deviation parameter among the local parameter of the local models trained by the sample sets comprises:

determining whether the local correction parameter of each of the local models is the at least one deviation parameter through an error detection model.

7 . The optimizing method of distributed training according to claim 6 , wherein the error detection model is obtained through an autoencoder training a plurality of normal correction parameters.

8 . The optimizing method of distributed training according to claim 1 , further comprising:

alerting the at least one deviation parameter.

9 . A master computing apparatus, comprising:

a memory, storing a code; and

a processor, coupled to the memory and loading the code to execute:

obtaining a local parameter corresponding to a plurality of sample sets, wherein the local parameter corresponding to each of the sample set are generated by training a local model using one of the sample sets and a global parameter of a global model;

determining at least one deviation parameter among the local parameter of a plurality of local models trained by the sample sets, wherein a distribution of the at least one deviation parameter is far from a distribution of other local parameters, and the local parameter of the local models is used to update the global parameter of the global model;

deleting the at least one deviation parameter from the local parameter of the local models trained by the sample sets; and

updating the global parameter of the global model with other local parameters other than the at least one deviation parameter;

wherein the local parameter of each of the local models comprises a local correction parameter, the global parameter comprises a global correction parameter, the global correction parameter is obtained by a weighted operation based on the local correction parameter, and the processor further executes:

determining the at least one deviation parameter according to an operation weight used by the local correction parameter of each of the local models in the weighted operation.

10 . The master computing apparatus according to claim 9 , wherein the processor is further configured to execute:

comparing the operation weight used by the local correction parameter of each of the local models in the weighted operation with a threshold deviation;

using the local correction parameter of one of the local models as the at least one deviation parameter in response to the operation weight used by the local correction parameter of the one of the local models in the weighted operation being less than the threshold deviation; and

prohibiting the local correction parameter of one of the local models as the at least one deviation parameter in response to the operation weight used by the local correction parameter of the one of the local models in the weighted operation not being less than the threshold deviation.

11 . The master computing apparatus according to claim 10 , wherein the processor is further configured to execute:

determining a probability distribution of the operation weight used by the local correction parameter of the local models in the weighted operation; and

taking a value corresponding to a deviation quantile in the probability distribution as the threshold deviation.

12 . The master computing apparatus according to claim 11 , wherein the probability distribution is a t distribution.

13 . The master computing apparatus according to claim 9 , wherein the local correction parameter is a gradient value determined based on a gradient descent method.

14 . The master computing apparatus according to claim 9 , wherein the local parameter of each of the local models comprises a local correction parameter, the global parameter comprises a global correction parameter, the global correction parameter is obtained by a weighted operation based on the local correction parameter, and the processor further executes:

determining whether the local correction parameter of each of the local models is the at least one deviation parameter through an error detection model.

15 . The master computing apparatus according to claim 14 , wherein the error detection model is obtained through an autoencoder using a plurality of normal correction parameters.

16 . The master computing apparatus according to claim 9 , wherein the processor is further configured to execute:

alerting the at least one deviation parameter.