IP Library › Granted Patent US 12,596,959
Granted Patent B2
US 12,596,959 · App. 18/593,496 · Granted Apr 7, 2026

Method for collaborative machine learning

Inventors: Alice Dethise (Stuttgart, DE); Ruichuan Chen (Stuttgart, DE); Istemi Ekin Akkus (Stuttgart, DE); Paarijaat Aditya (Stuttgart, DE); Antti Herman Koskela (Espoo, FI)
Assignee: Nokia Solutions and Networks Oy
G06N20/00G06F21/602
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,596,959
App. No.
18/593,496
Granted
Apr 7, 2026
Kind
B2
Abstract

A system comprising: at least one training unit for one or more data owners collaborating to the system for storing private data and at least one encrypted model, said training unit being implemented as a trusted execution environment; at least one aggregator unit for each model owner collaborating to the system for storing and executing code of a training algorithm, said aggregator unit being implemented as a trusted execution environment; at least one administration unit for controlling communication and synchronization between the at least one training unit and the at least one aggregator unit, said administration unit being implemented as a trusted execution environment; wherein the communication between the at least one training unit and the at least one aggregator unit is encrypted.

Claims (39)

1 . An apparatus comprising at least one training unit for one or more data owners collaborating to a system for storing private data and at least one encrypted model, said training unit being implemented as a trusted execution environment; said training unit being configured to communicate with at least one aggregator unit of the system; wherein the communication between the at least one training unit and the at least one aggregator unit is encrypted; and wherein at least one administration unit is configured to receive model gradients from the at least one training unit, generate masks for the gradients of each of the at least one training unit such that the sum of the masks equals to a predefined noise level, and transmit the masks to at least one corresponding unit of the at least one training unit; said at least one administration unit being configured for controlling communication and synchronization between the at least one training unit and the at least one aggregator, and said at least one administration unit being implemented in the trusted executed environment.

2 . The apparatus according to claim 1 , wherein the at least one training unit is configured to compute the gradients based on the private data of one or more respective data owners using said at least one encrypted model.

3 . The apparatus according to claim 2 , wherein the at least one training unit is configured to send the gradients to the at least one administration unit and receive its corresponding mask for the gradients from the at least one administration unit.

4 . The apparatus according to claim 3 , wherein the at least one training unit is configured to apply said corresponding mask received from the administration unit to the model gradient and transmit a masked model gradient to the at least one aggregator unit.

5 . The apparatus according to claim 1 , wherein the at least one training unit is configured to receive information indicating a subset of gradients to use from the administration unit.

6 . The apparatus according to claim 1 , wherein the at least one training unit is configured to randomly select a subset of gradients to use.

7 . The apparatus according to claim 1 , wherein the at least one training unit is prevented from interacting with any data unit outside the training unit, except via one or more predetermined interfaces of the system.

8 . An apparatus comprising at least one aggregator unit for each model owner collaborating to a system for storing and executing code of a training algorithm, said aggregator unit being implemented as a trusted execution environment; said aggregator unit being configured to communicate with at least one training unit of the system; wherein the communication between the at least one training unit and the at least one aggregator unit is encrypted; and wherein at least one administration unit is configured to receive model gradients from the at least one training unit, generate masks for the gradients of each of the at least one training unit such that the sum of the masks equals to a predefined noise level, and transmit the masks to at least one corresponding unit of the at least one training unit; said at least one administration unit being configured for controlling communication and synchronization between the at least one training unit and the at least one aggregator, and said at least one administration unit being implemented in the trusted executed environment.

9 . The apparatus according to claim 8 , where the at least one aggregator unit is configured to receive a masked model gradient from the at least one training unit and update the model based on the masked model gradients.

10 . The apparatus according to claim 8 , wherein the at least one aggregator unit is prevented from interacting with any data unit outside the aggregator unit, except via one or more predetermined interfaces of the system.

11 . An apparatus comprising at least one administration unit for controlling communication and synchronization between at least one training unit and at least one aggregator unit, said administration unit being implemented as a trusted execution environment; wherein the at least one administration unit is configured to receive gradients from the at least one training unit, generate masks for the gradients of each of the at least one training unit such that the sum of the masks equals to a predefined noise level, and transmit the masks to at least one corresponding unit of the at least one training unit.

12 . The apparatus according to claim 11 , wherein the masks are computed as

∑

i

=

0

n

m

i

~

N

⁡

(

0

,

σ

2

⁢

C

2

⁢

I

)

~

ξ

where m i is the mask sent to the i-th training enclave, σ is the privacy parameter, and C is the gradient clipping bound.

13 . The apparatus according to claim 11 , the administration unit is configured to receive a sample of norms of respective gradients from one or more training units; create an approximation of the distribution of gradient norms across all the gradients; select a threshold value for a clipping bound based on a predefined distribution-dependent target value; and send said threshold value to said one or more training units.

14 . The apparatus according to claim 11 , wherein the administration unit is configured to compute a noise value at a present time instance, wherein a weighted noise value of a previous time instance is subtracted from a new noise value introduced at the present time instance.

15 . The apparatus according to claim 11 , wherein the administration unit is configured to provide one or more training units with information indicating a subset of gradients to use.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2025
From: ANDRE B DETHISE, ARNAUD; CHEN, RUICHUAN; EKIN AKKUS, ISTEMI; ADITYA, PAARIJAAT
To: NOKIA SOLUTIONS AND NETWORKS GMBH & CO. KG
Reel/Frame 071016/0690 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2025
From: HERMAN KOSKELA, ANTTI
To: NOKIA SOLUTIONS AND NETWORKS OY
Reel/Frame 071016/0697 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2025
From: NOKIA SOLUTIONS AND NETWORKS GMBH & CO. KG
To: NOKIA SOLUTIONS AND NETWORKS OY
Reel/Frame 071016/0700 →
Priority Claims (1)
FI 20235286 · Mar 10, 2023 · national
Continuity (1)
Related Publication 20240303548A1 · Sep 12, 2024
References Cited (70)
US 11526745B2 · Sheller · 2022 [cited by examiner]
US 11748661B2 · Jing · 2023 [cited by examiner]
US 20170372226A1 · Costa · 2017 [cited by examiner]
US 20200082270A1 · Gu et al. · 2020 [cited by applicant]
US 20200250321A1 · Wu · 2020 [cited by examiner]
US 20200327250A1 · Wang et al. · 2020 [cited by applicant]
US 20210073677A1 · Peterson et al. · 2021 [cited by applicant]
US 20210150037A1 · Radhakrishnan · 2021 [cited by examiner]
US 20210158216A1 · Du et al. · 2021 [cited by applicant]
US 20210374605A1 · Qian et al. · 2021 [cited by applicant]
US 20210383280A1 · Shaloudegi · 2021 [cited by applicant]
US 20220108226A1 · Yu et al. · 2022 [cited by applicant]
US 20220261697A1 · Chopra · 2022 [cited by examiner]
US 20220294606A1 · Norrman et al. · 2022 [cited by applicant]
US 20220374762A1 · Radhakrishnan et al. · 2022 [cited by applicant]
US 20230039182A1 · Cheng · 2023 [cited by examiner]
US 20240256900A1 · Weng · 2024 [cited by examiner]
US 20240362361A1 · Shao · 2024 [cited by examiner]
US 20250315732A1 · Ustyuzhanin · 2025 [cited by examiner]
EP 4083868A1 · 2022 [cited by applicant]
Tramer et al., “Stealing Machine Learning Models via Prediction APIs”, Proceedings of the 25th USENIX Security Symposium, Aug. 10-12, 2016, pp. 601-618. [cited by applicant]
Orekondy et al., “Knockoff Nets: Stealing Functionality of Black-Box Models”, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 16-20, 2019, pp. 4954-4963. [cited by applicant]
Hitaj et al., “Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning”, Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, Oct. 30-Nov. 3, 2017, pp. 603-618. [cited by applicant]
Mandal et al., “PrivFL: Practical Privacy-preserving Federated Regressions on High-dimensional Data over Mobile Networks”, Proceedings of the ACM SIGSAC Conference on Cloud Computing Security Workshop, Nov. 11, 2019, pp… [cited by applicant]
Phong et al., “Privacy-Preserving Deep Learning via Additively Homomorphic Encryption”, IEEE Transactions on Information Forensics and Security, vol. 13, No. 05, May 2018, pp. 1333-1345. [cited by applicant]
Hsieh et al., “Gaia: Geo-Distributed Machine Learning Approaching LAN Speeds”, Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI), Mar. 27-29, 2017, pp. 629-647. [cited by applicant]
Bagdasaryan et al., “Differential Privacy Has Disparate Impact on Model Accuracy”, Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS), 2019, pp. 1-10. [cited by applicant]
Du et al., “Privacy-Preserving Multivariate Statistical Analysis: Linear Regression and Classification”, Proceedings of the SIAM International Conference on Data Mining (SDM), 2004, pp. 222-233. [cited by applicant]
Bonawitz et al., “Practical Secure Aggregation for Privacy-Preserving Machine Learning”, Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, Oct. 30-Nov. 3, 2017, pp. 1175-1191. [cited by applicant]
Takabi et al., “Privacy Preserving Multi-party Machine Learning with Homomorphic Encryption”, Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS), Dec. 5-10, 2016, pp. 1-5. [cited by applicant]
Gregor et al., “Trust Management as a Service: Enabling Trusted Execution in the Face of Byzantine Stakeholders”, 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Jun. 29-Jul. 2, … [cited by applicant]
Volos et al., “Graviton: Trusted Execution Environments on GPUs”, 13th USENIX Symposium on Operating Systems Design and Implementation, Oct. 8-10, 2018, pp. 681-696. [cited by applicant]
Tramer et al., “Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware”, arxiv, Jun. 8, 2018, pp. 1-15. [cited by applicant]
Ohrimenko et al., “Oblivious Multi-Party Machine Learning on Trusted Processors”, Proceedings of the 25th USENIX Security Symposium, Aug. 10-12, 2016, pp. 619-636. [cited by applicant]
Tianjian, “Federated Learning Inside: Introduction to Ant Financial's shared learning”, Published in Federated Learning, Jul. 2, 2017, 8 pages. [cited by applicant]
Hunt et al., “Chiron: Privacy-preserving Machine Learning as a Service”, arxiv, Mar. 15, 2018, pp. 1-15. [cited by applicant]
Arnautov et al., “SCONE: Secure Linux Containers with Intel SGX”, Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, Nov. 2-4, 2016, pp. 689-703. [cited by applicant]
Bayerl et al., “Offline Model Guard: Secure and Private ML on Mobile Devices”, Proceedings of the 23rd Conference on Design, Automation and Test in Europe, Mar. 2020, pp. 460-465. [cited by applicant]
“Ping An: Security Technology Reduces Data Silos”, Intel, Retrieved on Feb. 26, 2024, Webpage available at : https://www.intel.com/content/www/us/en/customer-spotlight/stories/ping-an-sgx-customer-story.html. [cited by applicant]
Wu et al., “A Privacy Preserving ML System”, Electronics Engineering & Computer Sciences, UC Berkeley, pp. 1-12. [cited by applicant]
Jia et al., “Preserving Model Privacy for Machine Learning in Distributed Systems”, IEEE Transactions on Parallel and Distributed Systems, vol. 29, No. 8, Aug. 2018, pp. 1808-1822. [cited by applicant]
“Pysyft, Pytorch and Intel SGX: Secure Aggregation on Trusted Execution Environments”, Openmined, Retrieved on Feb. 26, 2024, Webpage available at : https://blog.openmined.org/pysyft-pytorch-intel-sgx/. [cited by applicant]
Hynes et al., “Efficient Deep Learning on Multi-Source Private Data”, arxiv, Jul. 17, 2018, 7 pages. [cited by applicant]
Samadi et al., “Paraprox: pattern-based approximation for data parallel applications”, ACM SIGARCH Computer Architecture News, Mar. 1-5, 2014, pp. 35-50. [cited by applicant]
Lee et al., “On Model Parallelization and Scheduling Strategies for Distributed Machine Learning”, Advances in Neural Information Processing Systems 27, Dec. 8-13, 2014, pp. 1-9. [cited by applicant]
Abadi et al., “Deep Learning with Differential Privacy”, Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, Oct. 24-28, 2016, pp. 308-318. [cited by applicant]
Li et al., “Federated Learning: Challenges, Methods, and Future Directions”, IEEE Signal Processing Magazine, vol. 37, No. 03, May 2020, pp. 50-60. [cited by applicant]
Fereidooni et al., “SAFELearn: Secure Aggregation for private FEderated Learning”, IEEE Security and Privacy Workshops (SPW), 2021, pp. 56-62. [cited by applicant]
Zhang et al., “Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning”, Proceedings of the ACM Symposium on Cloud Computing, Nov. 1-4, 2021, pp. 546-561. [cited by applicant]
Mo et al., “SoK: Machine Learning with Confidential Computing”, arxiv, Aug. 22, 2022, pp. 1-18. [cited by applicant]
Yu et al., “Differentially Private Model Publishing for Deep Learning”, IEEE Symposium on Security and Privacy (SP), May 19-23, 2019, pp. 332-349. [cited by applicant]
Du et al., “Dynamic Differential-Privacy Preserving SGD”, arXiv, Jan. 17, 2022, 16 pages. [cited by applicant]
Haruki et al., “Gradient Noise Convolution (GNC): Smoothing Loss Function for Distributed Large-Batch SGD”, arXiv, Jun. 26, 2019, pp. 1-19. [cited by applicant]
Dupuy et al., “An Efficient DP-SGD Mechanism for Large Scale NLU Models”, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 23-27, 2022, pp. 4118-4122. [cited by applicant]
Fu et al., “Adap DP-FL: Differentially Private Federated Learning with Adaptive Noises”, IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Dec. 9-11, 2022, pp. 656-… [cited by applicant]
Wang et al., “DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERM”, Proceedings of Machine Learning Research, vol. 107, Jul. 20-24, 2020, pp. 328-351. [cited by applicant]
Zhou et al., “Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification”, arXiv, Apr. 23, 2021, pp. 1-35. [cited by applicant]
Lin et al., “Understanding adaptive gradient clipping in DP-SGD, empirically”, International Jounal of Intelligent Systems, vol. 37, No. 11, Aug. 25, 2022, pp. 9674-9700. [cited by applicant]
Li et al., “Differentially Private Federated Learning in Edge Networks: The Perspective of Noise Reductiony”, IEEE Network, vol. 36, No. 05, Sep./Oct. 2022, pp. 167-172. [cited by applicant]
Guo et al., “Topology-Aware Differential Privacy for Decentralized Image Classification”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, No. 06, Jun. 2022, pp. 4016-4027. [cited by applicant]
Cyffers et al., “Privacy Amplification by Decentralization”, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, vol. 151, Mar. 28-30, 2022, 20 pages. [cited by applicant]
Jiang et al., “Dordis: Efficient Federated Learning with Dropout-Resilient Differential Privacy”, arXiv, Nov. 10, 2023, 21 pages. [cited by applicant]
“DCsv3 and DCdsv3-series”, Microsoft, Retrieved on Feb. 25, 2024, Webpage available at : https://learn.microsoft.com/en-us/azure/virtual-machines/dcv3-series. [cited by applicant]
De et al., “Unlocking High-Accuracy Diferentially Private Image Classifcation through Scale”, arXiv, Jun. 16, 2022, pp. 1-34. [cited by applicant]
Koskela, “Analyze the correction formula tightly”, Nokia Bell Labs, 2022, pp. 1-3. [cited by applicant]
“Gaussian Mechanism with Noise Correction”, 2022, pp. 1-14. [cited by applicant]
Luo et al., “SVFL: Efficient Secure Aggregation and Verification for Cross-Silo Federated Learning”, IEEE Transactions on Mobile Computing, vol. 23, No. 01, Jan. 2024, pp. 850-864. [cited by applicant]
Tenison et al., “Gradient-Masked Federated Optimization”, ICLR, Workshop on Distributed and Private Machine Learning (DPML), 2021, pp. 1-7. [cited by applicant]
Office action received for corresponding Finnish Patent Application No. 20235286, dated Jul. 20, 2023, 11 pages. [cited by applicant]
Extended European Search Report received for corresponding European Patent Application No. 24157418.5, dated Jun. 24, 2024, 10 pages. [cited by applicant]