Training machine learning models to predict characteristics of adverse events using intermittent data
Methods, systems, and apparatus for providing a ML model for inference, the ML model having been trained using a first set of training data to provide predictions associated with an adverse event, after training of the ML model, receiving data from one or more data sources, the data representative of characteristics relevant to predictions associated with the adverse event, providing a second set of training data, determining, by a trigger module, a trigger decision based on a set of signals at least partially determined from the second set of training data, the trigger decision indicating whether the ML model is to be one of updated and retrained based on the second set of training data, and selectively executing one of updating and retraining of the ML model using at least a portion of the second set of training data in response to the trigger decision.
1 . A computer-implemented method executed by one or more processors, and comprising:
providing a machine learning (ML) model for inference, the ML model having been trained using a first set of training data to provide predictions associated with an adverse climatological event in a geographic region;
after training of the ML model, receiving data from one or more data sources, the data representative of characteristics relevant to predictions associated with the adverse climatological event in the geographic region;
providing a second set of training data based on the data received from the one or more data sources;
determining, by a trigger module, a trigger decision based on a set of signals at least partially determined from the second set of training data, the trigger decision indicating whether the ML model is to be one of updated or retrained using at least a portion of the second set of training data, wherein the set of signals comprises a climatological signal representative of onset of an adverse climatological event in the geographic region; and
selectively executing one of updating or retraining of the ML model using at least a portion of the second set of training data in response to the trigger decision.
2 . The method of claim 1 , wherein the set of signals comprises one or more of a data statistics signal and a data ratio signal, the data statistics signal representing at least one statistical value determined for the second set of training data, the data ratio signal represents a ratio of updated training data in the second set of training data relative to the first set of training data.
3 . The method of claim 1 , wherein the set of signals further comprises a decision point signal, the decision point signal representative of a decision point in triggering one of updating or retraining of the ML model.
4 . The method of claim 1 , wherein the trigger decision is determined by:
providing an aggregate trigger signal based on two or more trigger signals determine based on the set of signals; and
comparing the aggregate signal to a threshold.
5 . The method of claim 1 , wherein the trigger decision is determined by inputting the set of signals to a trigger ML model that provides the trigger decision as output.
6 . The method of claim 1 , wherein, after updating or retraining of the ML model, a metric determined for the ML model exceeds a threshold improvement relative to the metric determined for the ML model prior to updating or retraining of the ML model.
7 . The method of claim 1 , wherein the ML model is an ensemble model and updating comprises adding a model to the ensemble model based on the second set of training data.
8 . A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
providing a machine learning (ML) model for inference, the ML model having been trained using a first set of training data to provide predictions associated with an adverse climatological event in a geographic region;
after training of the ML model, receiving data from one or more data sources, the data representative of characteristics relevant to predictions associated with the adverse climatological event in the geographic region;
providing a second set of training data based on the data received from the one or more data sources;
determining, by a trigger module, a trigger decision based on a set of signals at least partially determined from the second set of training data, the trigger decision indicating whether the ML model is to be one of updated or retrained using at least a portion of the second set of training data, wherein the set of signals comprises a climatological signal representative of onset of an adverse climatological event in the geographic region; and
selectively executing one of updating or retraining of the ML model using at least a portion of the second set of training data in response to the trigger decision.
9 . The non-transitory computer-readable storage medium of claim 8 , wherein the set of signals comprises one or more of a data statistics signal and a data ratio signal, the data statistics signal representing at least one statistical value determined for the second set of training data, the data ratio signal represents a ratio of updated training data in the second set of training data relative to the first set of training data.
10 . The non-transitory computer-readable storage medium of claim 8 , wherein the set of signals further comprises a decision point signal, the decision point signal representative of a decision point in triggering one of updating or retraining of the ML model.
11 . The non-transitory computer-readable storage medium of claim 8 , wherein the trigger decision is determined by:
providing an aggregate trigger signal based on two or more trigger signals determine based on the set of signals; and
comparing the aggregate signal to a threshold.
12 . The non-transitory computer-readable storage medium of claim 8 , wherein the trigger decision is determined by inputting the set of signals to a trigger ML model that provides the trigger decision as output.
13 . The non-transitory computer-readable storage medium of claim 8 , wherein, after updating or retraining of the ML model, a metric determined for the ML model exceeds a threshold improvement relative to the metric determined for the ML model prior to updating or retraining of the ML model.
14 . The non-transitory computer-readable storage medium of claim 8 , wherein the ML model is an ensemble model and updating comprises adding a model to the ensemble model based on the second set of training data.
15 . A system, comprising:
a computing device; and
a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations comprising:
providing a machine learning (ML) model for inference, the ML model having been trained using a first set of training data to provide predictions associated with an adverse climatological event in a geographic region;
after training of the ML model, receiving data from one or more data sources, the data representative of characteristics relevant to predictions associated with the adverse climatological event in the geographic region;
providing a second set of training data based on the data received from the one or more data sources;
determining, by a trigger module, a trigger decision based on a set of signals at least partially determined from the second set of training data, the trigger decision indicating whether the ML model is to be one of updated or retrained using at least a portion of the second set of training data, wherein the set of signals comprises a climatological signal representative of onset of an adverse climatological event in the geographic region; and
selectively executing one of updating or retraining of the ML model using at least a portion of the second set of training data in response to the trigger decision.
16 . The system of claim 15 , wherein the set of signals comprises one or more of a data statistics signal and a data ratio signal, the data statistics signal representing at least one statistical value determined for the second set of training data, the data ratio signal represents a ratio of updated training data in the second set of training data relative to the first set of training data.
17 . The system of claim 15 , wherein the set of signals further comprises a decision point signal, the decision point signal representative of a decision point in triggering one of updating or retraining of the ML model.
18 . The system of claim 15 , wherein the trigger decision is determined by:
providing an aggregate trigger signal based on two or more trigger signals determine based on the set of signals; and
comparing the aggregate signal to a threshold.
19 . The system of claim 15 , wherein the trigger decision is determined by inputting the set of signals to a trigger ML model that provides the trigger decision as output.
20 . The system of claim 15 , wherein, after updating or retraining of the ML model, a metric determined for the ML model exceeds a threshold improvement relative to the metric determined for the ML model prior to updating or retraining of the ML model.
21 . The system of claim 15 , wherein the ML model is an ensemble model and updating comprises adding a model to the ensemble model based on the second set of training data.