Method of load forecasting via knowledge distillation, and an apparatus for the same
A server may obtain teacher artificial intelligence (AI) models from source base stations; obtain target traffic data from a target base station; obtain an integrated teacher prediction based on the target traffic data by integrating teacher prediction results of the teacher AI models based on teacher importance weights; obtain a student AI model that is trained to converge a student loss on the target traffic data; update the teacher importance weights to converge a teacher loss between a student prediction of the student AI model on the target traffic data, and the integrated teacher prediction of the teacher AI models on the target traffic data; update the student AI model based on the updated teacher importance weights being applied to the teacher prediction results of the teacher AI models; and predict a communication traffic load of the target base station using the updated student AI model.
1. A server for predicting future load, the server comprising:
at least one memory storing computer-readable instructions; and
at least one processor configured to execute the computer-readable instructions to:
obtain a plurality of teacher artificial intelligence (AI) models that are trained based on source traffic data from a plurality of source base stations;
obtain target traffic data from a target base station;
obtain an integrated teacher model based on the target traffic data and the plurality of teacher artificial intelligence (AI) models;
obtain a student AI model that is trained based on the target traffic data;
update the integrated teacher model based on a difference between a student prediction of the student AI model on the target traffic data, and an integrated teacher prediction of the integrated teacher model on the target traffic data;
update the student AI model based on the updated integrated teacher model; and
predict a communication traffic load of the target base station using the updated student AI model.
2. The server of claim 1 , wherein the at least one processor is further configured to:
obtain the integrated teacher model based on the target traffic data and the plurality of teacher artificial intelligence (AI) models, by integrating teach prediction results of the plurality of teacher AI models based on teacher importance weights;
split the target traffic data into a training data set and a validation data set;
obtain a distillation knowledge loss and a ground-truth loss based on the training data set of the target traffic data;
obtain the student AI model that is trained to converge a student loss including the distillation knowledge loss and the ground-truth loss; and
obtain the student prediction of the student AI model and the integrated teacher prediction of the integrated teacher model based on the validation data set of the target traffic data, to update the teacher importance weights.
3. The server of claim 1 , wherein the at least one processor is further configured to:
obtain the integrated teacher model based on the target traffic data and the plurality of teacher artificial intelligence (AI) models, by integrating teach prediction results of the plurality of teacher AI models based on teacher importance weights;
update the integrated teacher model by updating the teacher importance weights based on the difference between the student prediction of the student AI model and the integrated teacher prediction of the integrated teacher model.
4. The server of claim 3 , wherein the at least one processor is further configured to update the student AI model based on the integrated teacher model to which the updated teacher importance weights are applied.
5. The server of claim 1 , wherein the at least one processor is further configured to:
compute a distillation knowledge loss of the student AI model based on the difference between the integrated teacher prediction and the student prediction of the student AI model on the target traffic data;
compute a ground-truth loss of the student AI model based on a difference between the student prediction of the student AI model on the target traffic data and a ground-truth traffic load; and
obtain the student model that is trained to converge a student loss including the distillation knowledge loss and the ground-truth loss.
6. The server of claim 1 , wherein the at least one processor is further configured to:
determine whether a prediction accuracy on a further traffic load of the target base station over a present past time window, is lower than an accuracy threshold; and
in response to determining that the prediction accuracy is lower than the accuracy threshold, start to collect the target traffic data from the target base station, and train the student AI model based on the integrated teacher prediction of the integrated teacher model.
7. The server of claim 1 , wherein the at least one processor is further configured to:
split the target traffic data into a training data set and a validation data set; and
at each iteration, update teacher importance weights to be applied to combine the plurality of teacher models of the integrated teacher model, and the student AI model, via gradient descent to minimize a teacher loss of the integrated teacher model on the validation data set and a student loss of the student AI model on the training data set, respectively.
8. The server of claim 1 , wherein the at least one processor is further configured to:
adjust a spectrum allocated to the target base station based on the predicted communication traffic load of the target base station.
9. A method for predicting future load, the method comprising:
obtaining a plurality of teacher artificial intelligence (AI) models that are trained based on source traffic data from a plurality of source base stations;
obtaining target traffic data from a target base station;
obtaining an integrated teacher model based on the target traffic data and the plurality of teacher artificial intelligence (AI) models;
obtaining a student AI model that is trained based on the target traffic data;
updating the integrated teacher model based on a difference between a student prediction of the student AI model on the target traffic data, and an integrated teacher prediction of the integrated teacher model;
updating the student AI model based on the updated integrated teacher model; and
predicting a communication traffic load of the target base station using the updated student AI model.
10. The method of claim 9 , further comprising:
obtaining the integrated teacher model based on the target traffic data and the plurality of teacher artificial intelligence (AI) models, by integrating teach prediction results of the plurality of teacher AI models based on teacher importance weights;
splitting the target traffic data into a training data set and a validation data set;
obtaining a distillation knowledge loss and a ground-truth loss based on the training data set of the target traffic data;
obtaining the student AI model that is trained to converge a student loss including the distillation knowledge loss and the ground-truth loss; and
obtaining the student prediction of the student AI model and the integrated teacher prediction of the integrated teacher model based on the validation data set of the target traffic data, to update the teacher importance weights.
11. The method of claim 9 , further comprising:
obtaining the integrated teacher model based on the target traffic data and the plurality of teacher artificial intelligence (AI) models, by integrating teach prediction results of the plurality of teacher AI models based on teacher importance weights;
updating the integrated teacher model by updating the teacher importance weights based on the difference between the student prediction of the student AI model and the integrated teacher prediction of the integrated teacher model.
12. The method of claim 11 , further comprising:
updating the student AI model based on the integrated teacher model to which the updated teacher importance weights are applied.
13. The method of claim 9 , further comprising:
computing a distillation knowledge loss of the student AI model based on the difference between the integrated teacher prediction and the student prediction of the student AI model on the target traffic data;
computing a ground-truth loss of the student AI model based on a difference between the student prediction of the student AI model on the target traffic data and a ground-truth traffic load; and
obtaining the student model that is trained to converge a student loss including the distillation knowledge loss and the ground-truth loss.
14. The method of claim 9 , further comprising:
determining whether a prediction accuracy on a further traffic load of the target base station over a present past time window, is lower than an accuracy threshold; and
in response to determining that the prediction accuracy is lower than the accuracy threshold, starting to collect the target traffic data from the target base station, and train the student AI model based on the integrated teacher prediction of the integrated teacher model.
15. The method of claim 9 , further comprising:
splitting the target traffic data into a training data set and a validation data set; and
at each iteration, updating teacher importance weights to be applied to combine the plurality of teacher models of the integrated teacher model, and the student AI model, via gradient descent to minimize a teacher loss of the integrated teacher model on the validation data set and a student loss of the student AI model on the training data set, respectively.
16. The method of claim 9 , further comprising:
adjusting a spectrum allocated to the target base station based on the predicted communication traffic load of the target base station.
17. A non-transitory computer-readable storage medium storing instructions which, when executed by at least one processor, causes the at least one processor to perform a method for predicting future load:
obtaining a plurality of teacher artificial intelligence (AI) models that are trained based on source traffic data from a plurality of source base stations;
obtaining target traffic data from a target base station;
obtaining an integrated teacher model based on the target traffic data and the plurality of teacher artificial intelligence (AI) models;
obtaining a student AI model that is trained based on the target traffic data;
updating the integrated teacher model based on a difference between a student prediction of the student AI model on the target traffic data, and an integrated teacher prediction of the integrated teacher model AI models on the target traffic data;
updating the student AI model based on the updated integrated teacher model; and
predicting a communication traffic load of the target base station using the updated student AI model.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises:
computing a distillation knowledge loss of the student AI model based on a difference between the integrated teacher prediction and the student prediction of the student AI model on the target traffic data;
computing a ground-truth loss of the student AI model based on a difference between the student prediction of the student AI model on the target traffic data and a ground-truth traffic load; and
obtaining the student model that is trained to converge a student loss including the distillation knowledge loss and the ground-truth loss.
19. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises:
splitting the target traffic data into a training data set and a validation data set;
at each iteration, updating teacher importance weights to be applied to combine the plurality of teacher models of the integrated teacher model, and the student AI model via gradient descent to minimize a teacher loss of the integrated teacher model on the validation data set and a student loss of the student AI mode on the training data set, respectively, and
adjusting a spectrum allocated to the target base station based on the predicted communication traffic load of the target base station.
20. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises:
obtaining the integrated teacher model based on the target traffic data and the plurality of teacher artificial intelligence (AI) models, by integrating teach prediction results of the plurality of teacher AI models based on teacher importance weights;
splitting the target traffic data into a training data set and a validation data set;
obtaining a distillation knowledge loss and a ground-truth loss based on the training data set of the target traffic data;
obtaining the student AI model that is trained to converge a student loss including the distillation knowledge loss and the ground-truth loss; and
obtaining the student prediction of the student AI model and the integrated teacher prediction of the integrated teacher model based on the validation data set of the target traffic data, to update the teacher importance weights.