Granted Patent B2

US 12,591,807 · App. 18/149,682 · Granted Mar 31, 2026

Inventors: Arpan Mukherjee (West Bengal, IN); Georgios Kollias (White Plains, NY); Theodoros Salonidis (Wayne, PA); Shiqiang Wang (White Plains, NY)

Assignee: International Business Machines Corporation

G06N20/00

Abstract

Claims (72)

1 . A computer-implemented method for automatic adaptive client selection in federated learning, the method comprising:

sending, by a server in the federated learning, to all of clients in the federated learning, parameters of a machine learning model, wherein all of the clients compute respective gradients using the parameters;

receiving, by the server from all of the clients, sketches of the respective gradients, wherein the sketches are computed by all of the clients;

using, by the server, the sketches to compute similarity between all of the clients;

clustering, by the server, all of the clients, based on the similarity;

optimizing, by the server, a number of client clusters and a dimension of the sketches, subject to a constraint of memory consumption, a constraint of communication overhead, and a performance metric of the federated learning;

determining, by the server, a subset of the clients that send the respective gradients to the server, by selecting the clients from respective ones of the client clusters; and

aggregating, by the server, the respective gradients sent by the subset of the clients.

2 . The computer-implemented method of claim 1 , further comprising:

in response to determining that the machine learning model does not converge, iterating, by the server, steps of sending the parameters to all of the clients, receiving the sketches from all of the clients, using the sketches to compute the similarity, clustering all of the clients, optimizing the number of the client clusters and the dimension of the sketches, determining the subset of the clients, and aggregating the respective gradients sent by the subset of the clients.

3 . The computer-implemented method of claim 1 , for optimizing the number of the client clusters and the dimension of the sketches, further comprising:

measuring, by the server, the memory consumption, the communication overhead, and the performance metric; and

training and updating, by the server, a predictive model predicting the performance metric, based on measurements of the memory consumption, the communication overhead, and the performance metric.

4 . The computer-implemented method of claim 1 , for optimizing the number of the client clusters and the dimension of the sketches, further comprising:

selecting, by the server, an initial maximum number of the client clusters and an initial maximum dimension of the sketches, wherein the constraint of the memory consumption and the constraint of the communication overhead are not violated;

clustering, by the server, all of the clients, based on the initial maximum number of the client clusters and the initial maximum dimension of the sketches;

for each value within a range of values of the number of the client clusters, decreasing, by the server, values of the dimension of the sketches, until clustering is substantially different;

keeping, by the server, a value of the dimension of the sketches that produces similar clustering; and

choosing, by the server, among the values of the number of the client clusters, a value of the number of the client clusters that yields a best value of the performance metric, by using a predictive model predicting the performance metric, wherein the predictive model is trained and updated based on measurements of the memory consumption, the communication overhead, and the performance metric.

5 . The computer-implemented method of claim 1 , further comprising:

selecting, by the server, one client from each of the client clusters.

6 . The computer-implemented method of claim 1 , further comprising:

storing, by the server, the sketches; and

storing, by the server, the respective gradients.

7 . The computer-implemented method of claim 1 , wherein the sketches are vectors of floats with lower dimensions than the respective gradients.

8 . A computer program product for automatic adaptive client selection in federated learning, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors, the program instructions executable to:

send, by a server in the federated learning, to all of clients in the federated learning, parameters of a machine learning model, wherein all of the clients compute respective gradients using the parameters;

receive, by the server from all of the clients, sketches of the respective gradients, wherein the sketches are computed by all of the clients;

use, by the server, the sketches to compute similarity between all of the clients;

cluster, by the server, all of the clients, based on the similarity;

optimize, by the server, a number of client clusters and a dimension of the sketches, subject to a constraint of memory consumption, a constraint of communication overhead, and a performance metric of the federated learning;

determine, by the server, a subset of the clients that send the respective gradients to the server, by selecting the clients from respective ones of the client clusters; and

aggregate, by the server, the respective gradients sent by the subset of the clients.

9 . The computer program product of claim 8 , further comprising the program instructions executable to:

in response to determining that the machine learning model does not converge, iterate, by the server, steps of sending the parameters to all of the clients, receiving the sketches from all of the clients, using the sketches to compute the similarity, clustering all of the clients, optimizing the number of the client clusters and the dimension of the sketches, determining the subset of the clients, and aggregating the respective gradients sent by the subset of the clients.

10 . The computer program product of claim 8 , for optimizing the number of the client clusters and the dimension of the sketches, further comprising the program instructions executable to:

measure, by the server, the memory consumption, the communication overhead, and the performance metric; and

train and update, by the server, a predictive model predicting the performance metric, based on measurements of the memory consumption, the communication overhead, and the performance metric.

11 . The computer program product of claim 8 , for optimizing the number of the client clusters and the dimension of the sketches, further comprising the program instructions executable to:

select, by the server, an initial maximum number of the client clusters and an initial maximum dimension of the sketches, wherein the constraint of the memory consumption and the constraint of the communication overhead are not violated;

cluster, by the server, all of the clients, based on the initial maximum number of the client clusters and the initial maximum dimension of the sketches;

for each value within a range of values of the number of the client clusters, decrease, by the server, values of the dimension of the sketches, until clustering is substantially different;

keep, by the server, a value of the dimension of the sketches that produces similar clustering; and

choose, by the server, among the values of the number of the client clusters, a value of the number of the client clusters that yields a best value of the performance metric, by using a predictive model predicting the performance metric, wherein the predictive model is trained and updated based on measurements of the memory consumption, the communication overhead, and the performance metric.

12 . The computer program product of claim 8 , further comprising the program instructions executable to:

select, by the server, one client from each of the client clusters.

13 . The computer program product of claim 8 , further comprising program instructions executable to:

store, by the server, the sketches; and

store, by the server, the respective gradients.

14 . The computer program product of claim 8 , wherein the sketches are vectors of floats with lower dimensions than the respective gradients.

15 . A computer system for automatic adaptive client selection in federated learning, the computer system comprising one or more processors, one or more computer readable tangible storage devices, and program instructions stored on at least one of the one or more computer readable tangible storage devices for execution by at least one of the one or more processors, the program instructions executable to:

send, by a server in the federated learning, to all of clients in the federated learning, parameters of a machine learning model, wherein all of the clients compute respective gradients using the parameters;

receive, by the server from all of the clients, sketches of the respective gradients, wherein the sketches are computed by all of the clients;

use, by the server, the sketches to compute similarity between all of the clients;

cluster, by the server, all of the clients, based on the similarity;

optimize, by the server, a number of client clusters and a dimension of the sketches, subject to a constraint of memory consumption, a constraint of communication overhead, and a performance metric of the federated learning;

determine, by the server, a subset of the clients that send the respective gradients to the server, by selecting the clients from respective ones of the client clusters; and

aggregate, by the server, the respective gradients sent by the subset of the clients.

16 . The computer system of claim 15 , further comprising the program instructions executable to:

in response to determining that the machine learning model does not converge, iterate, by the server, steps of sending the parameters to all of the clients, receiving the sketches from all of the clients, using the sketches to compute the similarity, clustering all of the clients, optimizing the number of the client clusters and the dimension of the sketches, determining the subset of the clients, and aggregating the respective gradients sent by the subset of the clients.

17 . The computer system of claim 15 , for optimizing the number of the client clusters and the dimension of the sketches, further comprising the program instructions executable to:

measure, by the server, the memory consumption, the communication overhead, and the performance metric; and

train and update, by the server, a predictive model predicting the performance metric, based on measurements of the memory consumption, the communication overhead, and the performance metric.

18 . The computer system of claim 15 , for optimizing the number of the client clusters and the dimension of the sketches, further comprising the program instructions executable to:

select, by the server, an initial maximum number of the client clusters and an initial maximum dimension of the sketches, wherein the constraint of the memory consumption and the constraint of the communication overhead are not violated;

cluster, by the server, all of the clients, based on the initial maximum number of the client clusters and the initial maximum dimension of the sketches;

for each value within a range of values of the number of the client clusters, decrease, by the server, values of the dimension of the sketches, until clustering is substantially different;

keep, by the server, a value of the dimension of the sketches that produces similar clustering; and

choose, by the server, among the values of the number of the client clusters, a value of the number of the client clusters that yields a best value of the performance metric, by using a predictive model predicting the performance metric, wherein the predictive model is trained and updated based on measurements of the memory consumption, the communication overhead, and the performance metric.

19 . The computer system of claim 15 , further comprising the program instructions executable to:

select, by the server, one client from each of the client clusters.

20 . The computer system of claim 15 , wherein the sketches are vectors of floats with lower dimensions than the respective gradients.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (21)