Private vertical federated learning
A second set of data identifiers, comprising identifiers of data usable in federated model training by a second data owner, is received at a first data owner from the second data owner. An intersection set of data identifiers is determined at the first data owner. At the first data owner according to the intersection set of data identifiers, the data usable in federated model training is rearranged by the first data owner to result in a first training dataset. At the first data owner using the intersection set of data identifiers, the first training dataset, and a previous iteration of an aggregated set of model weights, a first partial set of model weights is computed. An updated aggregated set of model weights, comprising the first partial set of model weights and a second partial set of model weights from the second data owner, is received from an aggregator.
1. A computer-implemented method comprising:
receiving, at a first data owner from a second data owner, a second set of data identifiers, the second set of data identifiers comprising identifiers of data usable in federated model training by the second data owner;
determining, at the first data owner by comparing the second set of data identifiers with a first set of data identifiers, an intersection set of data identifiers, the first set of data identifiers comprising identifiers of data usable in federated model training by the first data owner, the intersection set of data identifiers consisting of data identifiers present in both the first set of data identifiers and the second set of data identifiers, wherein the intersection set of data identifiers is stored in ascending numerical order;
rearranging, at the first data owner according to the intersection set of data identifiers, the data usable in federated model training by the first data owner, the rearranging resulting in a first training dataset sorted into ascending order of data identifiers in the intersection set of data identifiers, the first training dataset comprising a set of labels;
performing a training iteration of a model by computing, at the first data owner, a first partial set of model weights, the first partial set of model weights computed using the intersection set of data identifiers, the first training dataset, and a previous iteration of an aggregated set of model weights computed by the first data owner and the second data owner in a previous training iteration of the model;
receiving, at the first data owner from an aggregator, an updated aggregated set of model weights, the updated aggregated set of model weights comprising the first partial set of model weights and a second partial set of model weights received at the aggregator from the second data owner, the second partial set of model weights computed at the second data owner during the training iteration of the model, wherein the updated aggregated set of model weights comprises a result of the training iteration of the model; and
receiving, at the first data owner from the aggregator subsequent to performance of a plurality of training iterations including the training iteration, a trained version of the model, the trained version of the model comprising a final aggregated set of model weights computed at the first data owner and the second data owner.
2. The computer-implemented method of claim 1 , wherein the second set of data identifiers is communicated from the second data owner through the aggregator to the first data owner.
3. The computer-implemented method of claim 1 , wherein the second set of data identifiers is communicated from the second data owner through the aggregator to the first data owner in homomorphically encrypted form, and wherein a decryption key usable to decrypt the second set of data identifiers is unavailable to the aggregator.
4. The computer-implemented method of claim 1 , wherein the intersection set of data identifiers comprises a homomorphically encrypted indicator vector.
5. The computer-implemented method of claim 4 , wherein the homomorphically encrypted indicator vector is determined by comparing a homomorphically encrypted form of the second set of data identifiers with a homomorphically encrypted form of the first set of data identifiers.
6. The computer-implemented method of claim 1 , wherein the intersection set of data identifiers is communicated from the first data owner through the aggregator to the second data owner.
7. The computer-implemented method of claim 1 , wherein the intersection set of data identifiers is communicated from the first data owner through the aggregator to the second data owner in homomorphically encrypted form, and wherein a decryption key usable to decrypt the intersection set of data identifiers is unavailable to the aggregator.
8. The computer-implemented method of claim 1 , wherein the previous iteration of the aggregated set of model weights is received at the first data owner from the aggregator in encrypted form, wherein a decryption key usable to decrypt the previous iteration of the aggregated set of model weights is available to the first data owner.
9. The computer-implemented method of claim 1 , wherein computing the first partial set of model weights comprises:
computing first partial weighted data, the first partial weighted data comprising data from the first training dataset weighted by the previous iteration of the aggregated set of model weights;
receiving, from the second data owner, second partial weighted data, the second partial weighted data comprising data from the second training dataset weighted by the previous iteration of the aggregated set of model weights;
computing a set of differences between the set of labels and a sum of the first partial weighted data and the second partial weighted data; and
computing, using the set of differences and the previous iteration of the aggregated set of model weights, the first partial set of weights.
10. The computer-implemented method of claim 9 , wherein the second partial weighted data is communicated from the second data owner through the aggregator to the first data owner in homomorphically encrypted form, and wherein a decryption key usable to decrypt the second partial weighted data is unavailable to the aggregator.
11. The computer-implemented method of claim 9 , wherein the set of differences is computed using a homomorphically encrypted form of the first partial weighted data and a homomorphically encrypted form of the second partial weighted data, and the set of differences is in homomorphically encrypted form.
12. The computer-implemented method of claim 9 , wherein the partial set of weights is computed using a homomorphically encrypted form of the set of differences, and the partial set of weights is in homomorphically encrypted form.
13. A computer program product for vertical federated learning, the computer program product comprising:
one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the stored program instructions comprising:
program instructions to receive, at a first data owner from a second data owner, a second set of data identifiers, the second set of data identifiers comprising identifiers of data usable in federated model training by the second data owner;
program instructions to determine, at the first data owner by comparing the second set of data identifiers with a first set of data identifiers, an intersection set of data identifiers, the first set of data identifiers comprising identifiers of data usable in federated model training by the first data owner, the intersection set of data identifiers consisting of data identifiers present in both the first set of data identifiers and the second set of data identifiers, wherein the intersection set of data identifiers is stored in ascending numerical order;
program instructions to rearrange, at the first data owner according to the intersection set of data identifiers, the data usable in federated model training by the first data owner, the rearranging resulting in a first training dataset sorted into ascending order of data identifiers in the intersection set of data identifiers, the first training dataset comprising a set of labels;
program instructions to perform a training iteration of a model by computing, at the first data owner, a first partial set of model weights, the first partial set of model weights computed using the intersection set of data identifiers, the first training dataset, and a previous iteration of an aggregated set of model weights computed by the first data owner and the second data owner in a previous training iteration of the model;
program instructions to receive, at the first data owner from an aggregator, an updated aggregated set of model weights, the updated aggregated set of model weights comprising the first partial set of model weights and a second partial set of model weights received at the aggregator from the second data owner, the second partial set of model weights computed at the second data owner during the training iteration of the model, wherein the updated aggregated set of model weights comprises a result of the training iteration of the model; and
program instructions to receive, at the first data owner from the aggregator subsequent to performance of a plurality of training iterations including the training iteration, a trained version of the model, the trained version of the model comprising a final aggregated set of model weights computed at the first data owner and the second data owner.
14. The computer program product of claim 13 , wherein the second set of data identifiers is communicated from the second data owner through the aggregator to the first data owner.
15. The computer program product of claim 13 , wherein the second set of data identifiers is communicated from the second data owner through the aggregator to the first data owner in homomorphically encrypted form, and wherein a decryption key usable to decrypt the second set of data identifiers is unavailable to the aggregator.
16. The computer program product of claim 13 , wherein the intersection set of data identifiers comprises a homomorphically encrypted indicator vector.
17. The computer program product of claim 13 , wherein the stored program instructions are stored in the at least one of the one or more storage media of a local data processing system, and wherein the stored program instructions are transferred over a network from a remote data processing system.
18. The computer program product of claim 13 , wherein the stored program instructions are stored in the at least one of the one or more storage media of a server data processing system, and wherein the stored program instructions are downloaded over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system.
19. The computer program product of claim 13 , wherein the computer program product is provided as a service in a cloud environment.
20. A computer system comprising one or more processors, one or more computer-readable memories, and one or more computer-readable storage media, and program instructions stored on at least one of the one or more storage media for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:
program instructions to receive, at a first data owner from a second data owner, a second set of data identifiers, the second set of data identifiers comprising identifiers of data usable in federated model training by the second data owner;
program instructions to determine, at the first data owner by comparing the second set of data identifiers with a first set of data identifiers, an intersection set of data identifiers, the first set of data identifiers comprising identifiers of data usable in federated model training by the first data owner, the intersection set of data identifiers consisting of data identifiers present in both the first set of data identifiers and the second set of data identifiers, wherein the intersection set of data identifiers is stored in ascending numerical order;
program instructions to rearrange, at the first data owner according to the intersection set of data identifiers, the data usable in federated model training by the first data owner, the rearranging resulting in a first training dataset sorted into ascending order of data identifiers in the intersection set of data identifiers, the first training dataset comprising a set of labels;
program instructions to perform a training iteration of a model by computing, at the first data owner, a first partial set of model weights, the first partial set of model weights computed using the intersection set of data identifiers, the first training dataset, and a previous iteration of an aggregated set of model weights computed by the first data owner and the second data owner in a previous training iteration of the model;
program instructions to receive, at the first data owner from an aggregator, an updated aggregated set of model weights, the updated aggregated set of model weights comprising the first partial set of model weights and a second partial set of model weights received at the aggregator from the second data owner, the second partial set of model weights computed at the second data owner during the training iteration of the model, wherein the updated aggregated set of model weights comprises a result of the training iteration of the model; and
program instructions to receive, at the first data owner from the aggregator subsequent to performance of a plurality of training iterations including the training iteration, a trained version of the model, the trained version of the model comprising a final aggregated set of model weights computed at the first data owner and the second data owner.