Locality-aware compressor-decompressor for keeping prediction models up-to-date in resource constrained networks
A global prediction manager for generating predictions using data from data zones includes storage for storing a model repository comprising a global model set and a prediction manager. The prediction manager obtains a local model set from a data zone of the data zones indicating that the global model set is unacceptable; makes a determination that the local model set is acceptable; in response to the determination: distributes the local model set to at least one second data zone of the data zones; obtains compressed telemetry data, that was compressed using the local model set, from the data zone and the at least one second data zone; and generates a global prediction regarding a future operating condition of the data zones using: the compressed local telemetry data and the local model set.
1. A global prediction manager for generating predictions using data from data zones, comprising:
storage for storing a model repository comprising a global model set, wherein the global model set comprises a global prediction model and a global compression model; and
a prediction manager programmed to:
obtain a local model set from a data zone of the data zones indicating that the global model set is unacceptable, wherein the data zone determines, based on a predetermined threshold, that the global model set is unacceptable, wherein the data zone comprises a plurality of computing devices that provides computer implemented services to a client, wherein the local model set comprises a local prediction model and a local compression model;
based on the indication specifying that the global model set is unacceptable, obtain local telemetry data from a second data zone of the data zones, wherein the local telemetry data corresponds to a previous quality of computer implemented services provided by the second data zone;
make a determination, using the local telemetry data, that a prediction performed by the local model set is acceptable;
in response to the determination:
distribute the local model set to the second data zone;
obtain compressed local telemetry data from the second data zone and second compressed local telemetry data from the data zone, wherein the data zone and the second data zone compress telemetry data using the local compression model, wherein the local compression model is a lossy compression model; and
generate a global prediction regarding a future operating condition of the data zone and the second data zone to predict a future quality of the computer implemented services provided by the data zone and the second data zone using:
the compressed local telemetry data,
the second compressed local telemetry data, and
the local model set.
2. The global prediction manager of claim 1 , wherein the prediction manager is further programmed to:
modify an operation of the data zones based on the global prediction.
3. The global prediction manager of claim 1 , wherein the local prediction model is trained on only second local telemetry data of the data zone.
4. The global prediction manager of claim 3 , wherein the global prediction model is trained on both the second local telemetry data of the data zone and the local telemetry data of the second data zone.
5. The global prediction manager of claim 1 , wherein making the determination that the local model set is acceptable further comprises:
obtaining an operating condition associated with the local telemetry data;
obtaining a local prediction of the operating condition using:
the local model set, and
the local telemetry data;
obtaining the compressed second local telemetry data using:
the local model set, and
the local telemetry data;
obtaining decompressed local telemetry data using the compressed local telemetry data;
making the determination by analyzing:
the local telemetry data,
the decompressed local telemetry data,
the operating condition, and
the local prediction of the operating condition.
6. The global prediction manager of claim 1 , wherein the local compression model is trained on only second local telemetry data of the data zone.
7. The global prediction manager of claim 6 , wherein the global compression model is trained on both the second local telemetry data of the data zone and the local telemetry data of the second data zone.
8. A method for generating predictions using data from data zones, comprising:
obtaining a local model set from a data zone of the data zones indicating that a global model set using to generate the predictions is unacceptable, wherein the data zone determines, based on a predetermined threshold, that the global model set is unacceptable, wherein the data zone comprises a plurality of computing devices that provides computer implemented services to a client, wherein the local model set comprises a local prediction model and a local compression model, wherein the global set model comprises a global prediction model and a global compression model;
based on the indication specifying that the global model set is unacceptable, obtain local telemetry data from a second data zone of the data zones, wherein the local telemetry data corresponds to a previous quality of computer implemented services provided by the second data zone;
making a determination, using the local telemetry data, that a prediction performed by the local model set is acceptable;
in response to the determination:
distributing the local model set to the second data zone;
obtaining compressed local telemetry data from the second data zone and second compressed local telemetry data from the data zone, wherein the data zone and the second data zone compress telemetry data using the local compression model, wherein the local compression model is a lossy compression model; and
generating a global prediction regarding a future operating condition of the data zone and the second data zone to predict a future quality of the computer implemented services provided by the data zone and the second data zone using:
the compressed local telemetry data,
the second compressed local telemetry data, and
the local model set.
9. The method of claim 8 , further comprising:
modifying an operation of the data zones based on the global prediction.
10. The method of claim 8 , wherein the local prediction model is trained on only second local telemetry data of the data zone.
11. The method of claim 10 , wherein the global prediction model is trained on both the second local telemetry data of the data zone and the local telemetry data of the second data zone.
12. The method of claim 8 , wherein making the determination that the local model set is acceptable further comprises:
obtaining an operating condition associated with the local telemetry data;
obtaining a local prediction of the operating condition using:
the local model set, and
the local telemetry data;
obtaining the compressed local telemetry data using:
the local model set, and
the local telemetry data;
obtaining decompressed local telemetry data using the compressed local telemetry data;
making the determination by analyzing:
the local telemetry data,
the decompressed local telemetry data,
the operating condition, and
the local prediction of the operating condition.
13. The method of claim 8 , wherein the local compression model is trained on only second local telemetry data of the data zone.
14. The method of claim 13 , wherein the global compression model is trained on both the second local telemetry data of the data zone and the local telemetry data of the second data zone.
15. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for generating predictions using data from data zones, the method comprising:
obtaining a local model set from a data zone of the data zones indicating that a global model set using to generate the predictions is unacceptable, wherein the data zone determines, based on a predetermined threshold, that the global model set is unacceptable, wherein the data zone comprises a plurality of computing devices that provides computer implemented services to a client, wherein the local model set comprises a local prediction model and a local compression model, wherein the global set model comprises a global prediction model and a global compression model;
based on the indication specifying that the global model set is unacceptable, obtain local telemetry data from a second data zone of the data zones, wherein the local telemetry data corresponds to a previous quality of computer implemented services provided by the second data zone;
making a determination, using the local telemetry data, that a prediction performed by the local model set is acceptable;
in response to the determination:
distributing the local model set to the second data zone;
obtaining compressed local telemetry data from the second data zone and second compressed local telemetry data from the data zone, wherein the data zone and the second data zone compress telemetry data using the local compression model, wherein the local compression model is a lossy compression model; and
generating a global prediction regarding a future operating condition of the data zone and the second data zone to predict a future quality of the computer implemented services provided by the data zone and the second data zone using:
the compressed local telemetry data,
the second compressed local telemetry data, and
the local model set.
16. The non-transitory computer readable medium of claim 15 , wherein the method further comprises:
modifying an operation of the data zones based on the global prediction.
17. The non-transitory computer readable medium of claim 15 , wherein the local prediction model is trained on only second local telemetry data of the data zone.
18. The non-transitory computer readable medium of claim 17 , wherein the global prediction model is trained on both the second local telemetry data of the data zone and the second local telemetry data of the at least one second data zone.
19. The non-transitory computer readable medium of claim 15 , wherein making the determination that the local model set is acceptable further comprises:
obtaining an operating condition associated with the local telemetry data;
obtaining a local prediction of the operating condition using:
the local model set, and
the local telemetry data;
obtaining the compressed local telemetry data using:
the local model set, and
the local telemetry data;
obtaining decompressed local telemetry data using the compressed local telemetry data;
making the determination by analyzing:
the local telemetry data,
the decompressed local telemetry data,
the operating condition, and
the local prediction of the operating condition.
20. The non-transitory computer readable medium of claim 15 , wherein the local compression model is trained on only second local telemetry data of the data zone.