Data privacy protection method, server device and client device for federated learning
The disclosure provides a data privacy protection method, a server device, and a client device for federated learning. A public dataset is used to perform model training on a machine learning model by a server device to generate a gradient pool including multiple first gradients. The gradient pool and the machine learning model are received by a client device. The client device uses a local dataset to perform model training on the machine learning model to obtain a second gradient. A local gradient is selected from the first gradients in the gradient pool according to the second gradient using a differential privacy algorithm by the client device. An aggregated machine learning model is generated by performing model aggregation based on the local gradient by the server device.
1 . A data privacy protection method for federated learning comprising:
performing model training on a machine learning model by a server device using a public dataset to generate a gradient pool comprising a plurality of first gradients;
receiving the gradient pool and the machine learning model by a client device;
performing model training on the machine learning model by the client device using a local dataset to obtain a second gradient, wherein the second gradient is not transmitted to the server device;
selecting a local gradient from the first gradients in the gradient pool according to the second gradient by the client device using a differential privacy algorithm, wherein the selecting comprises:
calculating a distance between the second gradient and each of the first gradients, adding random interference noise to the distances to generate a plurality of interfered distances, picking a smallest one of the interfered distances, and selecting one of the first gradient corresponding to the smallest interfered distance as the local gradient; or
calculating a distance between the second gradient and each of the first gradients, calculating a probability of each of the first gradients using the distances where a reciprocal of each of the distances is the probability being selected, and sampling the local gradient according to the probabilities; and
performing model aggregation based on the local gradient by the server device to generate an aggregated machine learning model.
2 . The data privacy protection method for federated learning according to claim 1 , wherein selecting the local gradient from the first gradients in the gradient pool according to the second gradient by the client device using the differential privacy algorithm comprises:
mapping the second gradient to the local gradient of the first gradients by the client device using an exponential mechanism in the differential privacy algorithm.
3 . The data privacy protection method for federated learning according to claim 1 , wherein performing model aggregation based on the local gradient by the server device to generate the aggregated machine learning model comprises:
receiving the local gradient by the server device; and
performing model aggregation according to the local gradient and at least one other local gradient provided by at least one other client device by the server device to generate the aggregated machine learning model.
4 . The data privacy protection method for federated learning according to claim 3 further comprising:
transmitting the local gradient by the client device to the server device.
5 . The data privacy protection method for federated learning according to claim 1 , wherein performing model aggregation based on the local gradient by the server device to generate the aggregated machine learning model comprises:
receiving by the server device a plurality of local model weights generated based on the local gradient; and
performing model aggregation according to the local model weights and a plurality of other local model weights provided by at least one other client device by the server device to generate the aggregated machine learning model.
6 . The data privacy protection method for federated learning according to claim 5 further comprising:
updating the machine learning model by the client device using the local gradient to generate the local model weights; and
transmitting the local model weights by the client device to the server device.
7 . The data privacy protection method for federated learning according to claim 1 further comprising:
transmitting the aggregated machine learning model by the server device to the client device.
8 . The data privacy protection method for federated learning according to claim 1 , wherein performing model training on the machine learning model by the server device using the public dataset to generate the gradient pool comprising the first gradients comprises:
retrieving a plurality of sub-datasets from the public dataset by the server device; and
performing model training on the machine learning model by the server device using one of the sub-datasets to generate one of the first gradients.
9 . A server device comprising:
a transceiver;
a storage device storing a plurality of instructions; and
a processor coupled to the transceiver and the storage device, accessing and executing the instructions, and configured to:
perform model training on a machine learning model using a public dataset to generate a gradient pool comprising a plurality of first gradients;
transmit the gradient pool to a client device through the transceiver, wherein the client device is configured to perform model training on the machine learning model using a local dataset to obtain a second gradient which is not transmitted to the server device, and to select a local gradient from the first gradients in the gradient pool according to the second gradient using a differential privacy algorithm, wherein selecting the local gradient comprises:
calculating a distance between the second gradient and each of the first gradients, adding random interference noise to the distances to generate a plurality of interfered distances, picking a smallest one of the interfered distances, and selecting one of the first gradient corresponding to the smallest interfered distance as the local gradient; or
calculating a distance between the second gradient and each of the first gradients, calculating a probability of each of the first gradients using the distances where a reciprocal of each of the distances is the probability being selected, and sampling the local gradient according to the probabilities; and
perform model aggregation based on the local gradient to generate an aggregated machine learning model, wherein the local gradient is determined by the client device.
10 . The server device according to claim 9 , wherein the processor is further configured to:
receive the local gradient from the client device through the transceiver; and
perform model aggregation according to the local gradient and at least one other local gradient provided by at least one other client device to generate the aggregated machine learning model.
11 . The server device according to claim 9 , wherein the processor is configured to:
receive through the transceiver a plurality of local model weights generated based on the local gradient; and
perform model aggregation according to the local model weights and a plurality of other local model weights provided by at least one other client device to generate the aggregated machine learning model.
12 . The server device according to claim 9 , wherein the processor is configured to:
transmit the aggregated machine learning model to the client device through the transceiver.
13 . The server device according to claim 9 , wherein the processor is configured to:
retrieve a plurality of sub-datasets from the public dataset; and
perform model training on the machine learning model using one of the sub-datasets to generate one of the first gradients.
14 . A client device comprising:
a transceiver;
a storage device storing a plurality of instructions; and
a processor coupled to the transceiver and the storage device, accessing and executing the instructions, and configured to:
receive a gradient pool and a machine learning model from a server device through the transceiver, wherein the gradient pool comprises a plurality of first gradients;
perform model training on the machine learning model using a local dataset to obtain a second gradient, wherein the second gradient is not transmitted to the server device; and
select a local gradient from the first gradients in the gradient pool according to the second gradient using a differential privacy algorithm, wherein selecting the local gradient comprises:
calculating a distance between the second gradient and each of the first gradients, adding random interference noise to the distances to generate a plurality of interfered distances, picking a smallest one of the interfered distances, and selecting one of the first gradient corresponding to the smallest interfered distance as the local gradient; or
calculating a distance between the second gradient and each of the first gradients, calculating a probability of each of the first gradients using the distances where a reciprocal of each of the distances is the probability being selected, and sampling the local gradient according to the probabilities.
15 . The client device according to claim 14 , wherein the processor is configured to:
map the second gradient to the local gradient of the first gradients using an exponential mechanism in the differential privacy algorithm.
16 . The client device according to claim 14 , wherein the processor is configured to:
transmit the local gradient to the server device through the transceiver.
17 . The client device according to claim 14 , wherein the processor is configured to:
update the machine learning model using the local gradient to generate the local model weights; and
transmit the local model weights to the server device through the transceiver.