Granted Patent B2

US 12,591,785 · App. 18/408,775 · Granted Mar 31, 2026

Inventors: Alberto Conte (Massy, FR); Dario Bega (Munich, DE); Tejas Subramanya (Munich, DE); Abdelrahman Abdelkader (Munich, DE)

Assignee: Nokia Solutions and Networks Oy

G06N3/098H04L9/008G06N20/20

Abstract

Claims (35)

1 . An apparatus ( 120 , 320 , 520 ) for federated training, the apparatus comprising at least one processing core, at least one non-transitory memory including computer program code, the at least one memory and the computer program code being configured to, with the at least one processing core, to configure the apparatus for:

Transmitting a first implementation ( 22 , 42 ) of a data-processing model to a first distributed trainer ( 130 , 330 , 530 ), wherein the first implementation of the data-processing model comprises a first hidden part ( 221 , 421 ) and a first open part ( 222 , 422 ), wherein parameters of the first hidden part ( 221 , 421 ) are not accessible to the first distributed trainer, wherein the first distributed trainer has access to a first dataset embedded in a first remote device,

Transmitting a second implementation of the data-processing model to a second distributed trainer ( 130 , 330 , 530 ), wherein the second implementation ( 23 , 43 ) of the data-processing model comprises a second hidden part ( 231 , 431 ) and a second open part ( 232 , 432 ), wherein parameters of the second hidden part ( 231 ; 431 are not accessible to the second distributed trainer, wherein the second distributed trainer has access to a second dataset embedded in a second remote device,

Receiving a first training gradient ( 110 , 310 ) from the first distributed trainer ( 130 , 330 , 530 ) and a second training gradient ( 110 , 310 ) from the second distributed trainer, wherein the first gradient is computed using the first implementation ( 22 , 42 ) of the data-processing model and the first dataset, wherein the first gradient relates to the first open part ( 222 , 422 ) of the first implementation ( 22 , 42 ) of the data-processing model, wherein the second gradient is computed using the second implementation ( 23 , 43 ) of the data-processing model and the second dataset, wherein the second gradient relates to the second open part ( 232 , 432 ) of the second implementation ( 23 , 43 ) of the data-processing model,

Updating the data-processing model using the first gradient ( 110 , 310 ) and the second gradient,

Associating a first trust score to the first distributed trainer ( 130 , 330 , 530 ) and a second trust score to the second distributed trainer ( 130 , 330 , 530 ),

Determining the first hidden part ( 221 , 421 ) and the first open part ( 222 , 422 ) as a function of the first trust score and the second hidden part ( 231 , 431 ) and the second open part ( 232 , 432 ) as a function of the second trust score,

Wherein in response to the first trust score being greater than the second trust score, a number of parameters in the second hidden part ( 231 , 431 ) is larger than a number of parameters in the first hidden part ( 221 , 421 ).

2 . An apparatus ( 120 , 320 , 520 ) according to claim 1 , wherein the apparatus is further configured for:

after updating the data-processing model, iterating to the step of transmitting the first implementation ( 22 , 42 ) and the second implementation ( 23 , 43 ) of the updated data-processing model until the data-processing model fulfils a convergence criterion.

3 . An apparatus ( 120 , 320 , 520 ) according to claim 1 , wherein in response to the first trust score being greater than the second trust score, the second hidden part ( 231 , 431 ) comprises the first hidden part ( 221 , 421 ).

4 . An apparatus ( 120 , 320 , 520 ) according to claim 1 , wherein the data-processing model comprises a neural network, wherein the data-processing model comprises a plurality of layers of neurons and wherein the apparatus is further configured for determining the first hidden part ( 221 , 421 ) and the first open part ( 222 , 422 ) and for determining a limit layer in the set of layers of neurons, wherein the first hidden part ( 221 ; 421 ) comprises at least one layer preceding the limit layer in the set of layers of neurons, and wherein the open part ( 222 , 422 ) comprises the limit layer and at least one layer succeeding the limit layer in the set of layers of neurons.

5 . An apparatus ( 120 , 320 , 520 ) according to claim 1 , wherein the first hidden part ( 221 ; 421 ) of the data-processing model is a first executable file and the second hidden part ( 231 , 431 ) of the data-processing model is a second executable file.

6 . An apparatus ( 120 , 320 , 520 ) according to claim 5 , wherein transmitting the first implementation ( 22 , 42 ) of the data-processing model comprise further comprises encoding the first hidden part ( 221 , 421 ) of the data-processing model into the first executable file and the second hidden part ( 222 , 422 ) of the data-processing model into the second executable file.

7 . An apparatus ( 120 , 320 , 520 ) according to claim 1 , wherein the data-processing model is an encrypted version of an original data-processing model encrypted using a homomorphic encryption algorithm, wherein the data-processing model comprises a set of elementary blocks ( 411 , 412 , 413 , 414 ), wherein each of the set of elementary blocks ( 411 , 412 , 413 , 414 ) belongs either to the first hidden part ( 221 , 421 ) or to the first open part ( 222 , 422 ), wherein the apparatus is further configured for updating the data-processing models comprise means for updating each of the set of elementary blocks ( 411 , 412 , 413 , 414 ) in an intersection of the first ( 222 , 422 ) and second open parts ( 232 , 432 ) by combining the first and second training gradients.

8 . An apparatus ( 120 , 320 , 520 ) according to claim 1 , wherein the apparatus is further configured for transmitting a third implementation ( 24 , 44 ) of the data-processing model to a third distributed trainer, wherein parameters of the third implementation ( 24 , 44 ) are not accessible to the third distributed trainer, wherein the third distributed trainer has access to a third dataset embedded in a third remote device.

9 . An apparatus ( 120 , 320 , 520 ) according to claim 1 , comprising means wherein the apparatus is further configured for:

Transmitting a third implementation ( 21 , 41 ) of the data-processing model to a third distributed trainer, wherein parameters of the third implementation ( 21 , 41 ) are accessible to the third distributed trainer, wherein the third distributed trainer has access to a third dataset embedded in a third remote device,

Receiving a third training gradient from the third distributed trainer, wherein the third training gradient is computed using the third implementation ( 21 , 41 ) of the data-processing model and the third dataset, wherein the third training gradient relates to the parameters of the third implementation ( 21 , 41 ) of the data-processing model,

Updating the data-processing model using the third training gradient.

10 . An apparatus ( 520 ) according to claim 1 , further comprising wherein the apparatus is further configured for:

Transmitting each implementation within a first set of implementations of the data-processing model to a respective distributed trainer ( 530 ) within a first set of distributed trainers, wherein each respective distributed trainer ( 530 ) has access to a respective dataset embedded in a respective remote device,

Receiving a set of first elementary gradients from the first set of distributed trainers ( 530 ), wherein each of the first set of elementary gradients is computed by a respective distributed trainer in the first set of distributed trainers,

Computing an intermediate gradient from the set of first elementary gradients,

Updating the data-processing model using the first intermediate gradient and the second gradient.

11 . An apparatus ( 120 , 320 , 520 ) according to claim 10 , wherein each implementation in the first set of implementations has a common open part and a common hidden part, wherein the respective distributed trainer has access to parameters of the common open part and does not have access to the parameters of the common hidden part, wherein the common open part and the common hidden part are common to all implementations in the first set of implementations.

12 . A computer-implemented method for federated training, the method comprising the steps of:

Transmitting a first implementation ( 22 , 42 ) of a data-processing model to a first distributed trainer ( 130 , 330 , 530 ), wherein the first implementation of the data-processing model comprises a first hidden part ( 221 , 421 ) and a first open part ( 222 , 422 ), wherein parameters of the first hidden part ( 221 , 421 ) are not accessible to the first distributed trainer, wherein the first distributed trainer has access to a first dataset embedded in a first remote device,

Transmitting a second implementation of the data-processing model to a second distributed trainer ( 130 , 330 , 530 ), wherein the second implementation ( 23 , 43 ) of the data-processing model comprises a second hidden part ( 231 , 431 ) and a second open part ( 232 , 432 ), wherein parameters of the second hidden part ( 231 ; 431 ) are not accessible to the second distributed trainer, wherein the second distributed trainer has access to a second dataset embedded in a second remote device,

Receiving a first training gradient ( 110 , 310 ) from the first distributed trainer ( 130 , 330 , 530 ) and a second training gradient ( 110 , 310 ) from the second distributed trainer, wherein the first gradient is computed using the first implementation ( 22 , 42 ) of the data-processing model and the first dataset, wherein the first gradient relates to the first open part ( 222 , 422 ) of the first implementation ( 22 , 42 ) of the data-processing model, wherein the second gradient is computed using the second implementation ( 23 , 43 ) of the data-processing model and the second dataset, wherein the second gradient relates to the second open part ( 232 , 432 ) of the second implementation ( 23 , 43 ) of the data-processing model,

Updating the data-processing model using the first gradient ( 110 , 310 ) and the second gradient,

Associating a first trust score to the first distributed trainer ( 130 , 330 , 530 ) and a second trust score to the second distributed trainer ( 130 , 330 , 530 ),

Determining the first hidden part ( 221 , 421 ) and the first open part ( 222 , 422 ) as a function of the first trust score and the second hidden part ( 231 , 431 ) and the second open part ( 232 , 432 ) as a function of the second trust score,

Wherein in response to the first trust score being greater than the second trust score, a number of parameters in the second hidden part ( 231 , 431 ) is larger than a number of parameters in the first hidden part ( 221 , 421 ).

13 . A method according to claim 12 , further comprising the steps of: after updating the data-processing model, iterating to the step of transmitting the first implementation ( 22 , 42 ) and the second implementation ( 23 , 43 ) of the updated data-processing model until the data-processing model fulfils a convergence criterion.

Assignments (4)

Priority Claims (1)

Continuity (1)

References Cited (25)