Methods, apparatuses, and systems for updating service model based on privacy protection
A computer-implemented method, medium, and system are disclosed. One example method includes determining multiple model bases by multiple service parties. A respective local service model is constructed by each service party. Respective local training samples are processed by each service party using the respective local service model to determine respective gradient data corresponding to each model basis. The respective gradient data is sent to a server. In response to determining that the first model basis satisfies a gradient update condition, corresponding gradient data of the first model basis received from each service party are combined to obtain global gradient data corresponding to the first model basis. The global gradient data is sent to each service party. Reference parameters in local model basis corresponding to the first model basis are updated by each service party using the global gradient data to train the respective local service model.
1. A computer-implemented method, comprising:
determining, by a plurality of service parties, a plurality of model bases, wherein each model basis of the plurality of model bases comprises a plurality of corresponding reference parameters;
constructing, by each service party of the plurality of service parties, a respective local service model based on a respective combination of the plurality of model bases;
determining, by each service party, respective gradient data corresponding to each model basis including processing respective local training samples using the respective local service model;
sending, by each service party, the respective gradient data to a server;
determining, by the server, that a first model basis of the plurality of model bases satisfies a gradient update condition pre-stored in the server;
in response to determining that the first model basis satisfies the gradient update condition, combining, by the server, corresponding gradient data of the first model basis received from each service party to obtain global gradient data corresponding to the first model basis;
sending, by the server, the global gradient data corresponding to the first model basis to each service party; and
training, by each service party, the respective local service model including updating reference parameters in local model basis corresponding to the first model basis using the global gradient data.
2. The method according to claim 1 , wherein the gradient update condition comprises at least one of:
a quantity of gradient data of the first model basis received by the server from the plurality of service parties reaches a predetermined quantity; or
an update period has arrived.
3. The method according to claim 1 , wherein the server combines the corresponding gradient data of the first model basis received from each service party by one of: averaging, weighted averaging, or processing the corresponding gradient data arranged in a chronological order by using a pre-trained long short-term memory model.
4. The method according to claim 1 , wherein the respective local service model comprises a graph neural network, wherein each service party holds respective graph data constructed using local data, and wherein the respective graph data comprise heterogeneous graphs.
5. The method according to claim 1 , wherein the respective gradient data corresponding to each model basis comprises gradients respectively corresponding to reference parameters in each model basis.
6. The method according to claim 1 , wherein the respective combination of the plurality of model bases comprises at least one of:
a linear combination of the plurality of model bases; or
a network architecture search (NAS) based on the plurality of model bases.
7. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:
determining, by a plurality of service parties, a plurality of model bases, wherein each model basis of the plurality of model bases comprises a plurality of corresponding reference parameters;
constructing, by each service party of the plurality of service parties, a respective local service model based on a respective combination of the plurality of model bases;
determining, by each service party, respective gradient data corresponding to each model basis including processing respective local training samples using the respective local service model;
sending, by each service party, the respective gradient data to a server;
determining, by the server, that a first model basis of the plurality of model bases satisfies a gradient update condition pre-stored in the server;
in response to determining that the first model basis satisfies the gradient update condition, combining, by the server, corresponding gradient data of the first model basis received from each service party to obtain global gradient data corresponding to the first model basis;
sending, by the server, the global gradient data corresponding to the first model basis to each service party; and
training, by each service party, the respective local service model including updating reference parameters in local model basis corresponding to the first model basis using the global gradient data.
8. The non-transitory, computer-readable medium according to claim 7 , wherein the gradient update condition comprises at least one of:
a quantity of gradient data of the first model basis received by the server from the plurality of service parties reaches a predetermined quantity; or
an update period has arrived.
9. The non-transitory, computer-readable medium according to claim 7 , wherein the server combines the corresponding gradient data of the first model basis received from each service party by one of: averaging, weighted averaging, or processing the corresponding gradient data arranged in a chronological order by using a pre-trained long short-term memory model.
10. The non-transitory, computer-readable medium according to claim 7 , wherein the respective local service model comprises a graph neural network, wherein each service party holds respective graph data constructed using local data, and wherein the respective graph data comprise heterogeneous graphs.
11. The non-transitory, computer-readable medium according to claim 7 , wherein the respective gradient data corresponding to each model basis comprises gradients respectively corresponding to reference parameters in each model basis.
12. The non-transitory, computer-readable medium according to claim 7 , wherein the respective combination of the plurality of model bases comprises at least one of:
a linear combination of the plurality of model bases; or
a network architecture search (NAS) based on the plurality of model bases.
13. A computer-implemented system, comprising:
one or more computers; and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising:
determining, by a plurality of service parties, a plurality of model bases, wherein each model basis of the plurality of model bases comprises a plurality of corresponding reference parameters;
constructing, by each service party of the plurality of service parties, a respective local service model based on a respective combination of the plurality of model bases;
determining, by each service party, respective gradient data corresponding to each model basis including processing respective local training samples using the respective local service model;
sending, by each service party, the respective gradient data to a server;
determining, by the server, that a first model basis of the plurality of model bases satisfies a gradient update condition pre-stored in the server;
in response to determining that the first model basis satisfies the gradient update condition, combining, by the server, corresponding gradient data of the first model basis received from each service party to obtain global gradient data corresponding to the first model basis;
sending, by the server, the global gradient data corresponding to the first model basis to each service party; and
training, by each service party, the respective local service model including updating reference parameters in local model basis corresponding to the first model basis using the global gradient data.
14. The computer-implemented system according to claim 13 , wherein the gradient update condition comprises at least one of:
a quantity of gradient data of the first model basis received by the server from the plurality of service parties reaches a predetermined quantity; or
an update period has arrived.
15. The computer-implemented system according to claim 13 , wherein the server combines the corresponding gradient data of the first model basis received from each service party by one of: averaging, weighted averaging, or processing the corresponding gradient data arranged in a chronological order by using a pre-trained long short-term memory model.
16. The computer-implemented system according to claim 13 , wherein the respective local service model comprises a graph neural network, wherein each service party holds respective graph data constructed using local data, and wherein the respective graph data comprise heterogeneous graphs.
17. The computer-implemented system according to claim 13 , wherein the respective gradient data corresponding to each model basis comprises gradients respectively corresponding to reference parameters in each model basis.
18. The computer-implemented system according to claim 13 , wherein the respective combination of the plurality of model bases comprises at least one of:
a linear combination of the plurality of model bases; or
a network architecture search (NAS) based on the plurality of model bases.