Method and system with neural network model updating
A method of updating a neural network model by a terminal device, includes training a local model using a local data set collected by a terminal device to generate a trained local model; receiving, from a server, an independent identically distributed (i.i.d.) global data set, the i.i.d. global data set being a data set sampled for each class in a plurality of predefined classes; implementing the trained local model by inputting the i.i.d. global data set and transmitting final inference results of the implemented trained local model to the server; and receiving, from the server, a global model updated based on the final inference results of the inference.
1. A processor-implemented method, comprising:
training a local model using a local data set collected by a terminal device to generate a trained local model;
receiving, from a server, an independent identically distributed (i.i.d.) global data set, the i.i.d. global data set being a data set sampled in an independent identically distributed manner (i.i.d. manner) for each class in a plurality of predefined classes where sampling of the i.i.d. global data set in the i.i.d. manner includes sampling the i.i.d. global data set such that probability variable respectively corresponding to the classes have a same probability distribution and are independent of each other;
implementing the trained local model by inputting the i.i.d. global data set and transmitting final inference results of the implemented trained local model to the server; and
receiving, from the server, a global model updated based on the final inference results of the inference.
2. The method of claim 1 , wherein
the trained local model comprises a neural network trained to predict a class of the plurality of predefined classes corresponding to input data, and
the final inference results of the inference correspond to a hard label that indicates a class predicted for the i.i.d. global data set.
3. The method of claim 2 , wherein
the hard label has a smaller data size than a soft label that comprises information about probability values indicating respective probabilities for the i.i.d. global data set being classified into each of the plurality of predefined classes.
4. The method of claim 2 , wherein the global model is updated using a loss function having at least one of a ground truth (GT) label and the hard label as a variable.
5. The method of claim 2 , wherein,
receipt of the updated global model is performed even when the terminal device transmits only the hard label to the server and even when the global model and the trained local model have different structures.
6. The method of claim 1 , wherein the global model is updated in the server based on other final inference results received from other terminal devices in addition to the final inference results received from the terminal device.
7. The method of claim 1 , wherein the i.i.d. global data set has a smaller data size than the local data set.
8. The method of claim 1 , wherein the local data set comprises a data set sampled in a non-i.i.d. manner.
9. A processor-implemented method, comprising:
transmitting, to a plurality of terminal devices, an independent identically distributed (i.i.d.) global data set, the i.i.d. global data set being a data set sampled in an independent identically distributed manner (i.i.d. manner) for each class in a plurality of predefined classes where sampling of the i.i.d. global data set in the i.i.d. manner includes sampling the i.i.d. global data set such that probability variable respectively corresponding to the classes have a same probability distribution and are independent of each other;
receiving, from each of the plurality of terminal devices, final inference results of inference obtained by inputting the i.i.d. global data set to a local model trained in each of the terminal devices based on a corresponding local data set; and
updating a global model stored in a server based on the received final inference results of corresponding inferences from each of the plurality of terminal devices.
10. The method of claim 9 , wherein the final inference results of inference correspond to a hard label that indicates a class predicted for the i.i.d. global data set by the trained local model.
11. The method of claim 10 , wherein the updating of the global model is performed using a loss function having at least one of a ground truth (GT) label and the hard label as a variable.
12. The method of claim 11 , wherein the updating of the global model is performed to reduce a difference between at least one of the GT label and the hard label and a calculation result obtained by inputting the i.i.d. global data set to the global model.
13. The method of claim 9 , further comprising transmitting the updated global model to at least one of the terminal devices or another terminal device other than the plurality of terminal devices.
14. The method of claim 13 , wherein the updated global model is used as a pre-trained local model for the other terminal device.
15. A terminal device, comprising:
at least one memory configured to store a local model trained using a local data set collected by the terminal device; and
at least one processor configured to:
receive an independent identically distributed (i.i.d.) global data set, the i.i.d. global data set being a data set sampled in an independent identically distributed manner (i.i.d. manner) for each class in a plurality of predefined classes from a server where sampling of the i.i.d. global data set in the i.i.d. manner includes sampling the i.i.d. global data set such that probability variable respectively corresponding to the classes have a same probability distribution and are independent of each other;
implementing the trained local model by inputting the i.i.d. global data set and transmit final inference results of the implemented trained local model to the server; and
receive a global model updated based on the final inference results of the inference from the server.
16. The terminal device of claim 15 , wherein
the trained local model comprises a neural network trained to predict a class of the plurality of predefined classes corresponding to input data, and
the final inference results of the inference correspond to a hard label that indicates a class predicted for the i.i.d. global data set.
17. The terminal device of claim 16 , wherein the hard label has a smaller data size than a soft label that comprises information about probability values indicating respective probabilities for the i.i.d. global data set being classified into each of the plurality of predefined classes.
18. The terminal device of claim 16 , wherein the global model is updated using a loss function having at least one of a ground truth (GT) label and the hard label as a variable.
19. The terminal device of claim 16 , wherein, receipt of the global model is performed even when the terminal device transmits only the hard label to the server and even when the global model and the trained local model have different structures.
20. The terminal device of claim 15 , wherein the global model is updated in the server based on other final inference results of inference received from other terminal devices in addition to the final inference results of the inference received from the terminal device.