Robust multi-model event detection with unreliable sensors
Model selection is disclosed. Features used as inputs to models are scored in terms of importance and health. The importance and health scores are combined in order to generate a model score for each model. The model with a score above a threshold score is selected and deployed.
1 . A method performed by one or more processors of a near-edge node operating in communication with multiple sensors and far-edge nodes in an environment, the method comprising:
training models using sensor datasets collected from the multiple sensors associated with objects operating in the environment;
determining, for each of the models, a feature importance score for each feature of sensor outputs from the multiple sensors used as an input to each of the models, wherein each model receives a respective set of features from all possible features from the multiple sensors and a feature importance score for each feature of one model is different from a feature importance score for each feature of another model different from the one model;
determining, by each sensor, a feature health score for each feature of the sensor outputs based on deviations of the sensor outputs from expected normal operating behavior;
determining a model score for each of the models, wherein the model score is an aggregation of the feature importance scores of features, which are received by each model and with a feature health score is higher than a predetermined threshold, and the feature health scores of the corresponding features of the corresponding model such that degraded sensor features contribute reduced weight to the corresponding model score;
selecting a model from the models based on the corresponding model scores above a threshold score; and
deploying the selected model from the near-edge node to the far-edge nodes operating in the environment, wherein the selected model is automatically replaced when the feature health score of a feature used by the model falls below a threshold.
2 . The method of claim 1 , further comprising causing the far-edge nodes to generate inferences using the deployed selected model.
3 . The method of claim 2 , wherein features having a feature health score below a threshold score are omitted from an input to the deployed selected model.
4 . The method of claim 1 , further comprising determining the model score by generating a vector for each model, wherein the vector includes feature importance scores and feature health scores of a corresponding model.
5 . The method of claim 4 , wherein the feature importance scores are arranged in a first matrix and the feature health scores are arranged in a second matrix, wherein each vector is a dot product of a corresponding first matrix and a corresponding second matrix, wherein the vector includes a model score for each of the models.
6 . The method of claim 1 , further comprising redetermining the feature importance scores.
7 . The method of claim 1 , further comprising redetermining the feature health scores.
8 . The method of claim 1 , further comprising regenerating the model scores, wherein the deployed selected model is replaced with a new selected model if the new selected model has a higher model score than the deployed selected model.
9 . The method of claim 1 , wherein each of the models receives a subset of features selected from all features of all sensors.
10 . The method of claim 9 , wherein each of the subsets includes one or more features from at least one sensor.
11 . The method of claim 10 , wherein the model score is configured to reduce an influence of features that are faulty, noisy, or missing from inferences generated by the selected deployed model.
12 . A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors of a near-edge node operating in communication with multiple sensors and far-edge nodes in an environment, the instructions causing the processors to perform operations comprising:
training models using sensor datasets collected from the multiple sensors associated with objects operating in the environment;
determining, for each of the models, a feature importance score for each feature of sensor outputs from the multiple sensors used as an input to each of the models, wherein each model receives a respective set of features from all possible features from the multiple sensors and a feature importance score for each feature of one model is different from a feature importance score for each feature of another model different from the one model;
determining, by each sensor, a feature health score for each feature of the sensor outputs based on deviations of the sensor outputs from expected normal operating behavior;
determining a model score for each of the models, wherein the model score is an aggregation of the feature importance scores of features, which are received by each model and with a feature health score is higher than a predetermined threshold, and the feature health scores of the corresponding features of the corresponding model such that degraded sensor features contribute reduced weight to the corresponding model score;
selecting a model from the models based on the corresponding model scores above a threshold score; and
deploying the selected model from the near-edge node to the far-edge nodes operating in the environment, wherein the selected model is automatically replaced when the feature health score of a feature used by the model falls below a threshold.
13 . The non-transitory storage medium of claim 12 , further comprising causing the far-edge nodes to generate inferences using the deployed selected model.
14 . The non-transitory storage medium of claim 13 , wherein features having a feature health score below a threshold score are omitted from an input to the deployed selected model.
15 . The non-transitory storage medium of claim 12 , further comprising determining the model score by generating a vector for each model, wherein each vector includes feature importance scores and feature health scores of a corresponding model.
16 . The non-transitory storage medium of claim 15 , wherein the feature importance scores are arranged in a first matrix and the feature health scores are arranged in a second matrix, wherein each vector is a dot product of a corresponding first matrix and a second matrix, wherein the vector includes a model score for each of the models.
17 . The non-transitory storage medium of claim 12 , further comprising redetermining the feature importance scores and/or redetermining the feature health scores.
18 . The non-transitory storage medium of claim 12 , further comprising regenerating the model scores, wherein the deployed selected model is replaced with a new selected model if the new selected model has a higher model score than the deployed selected model.
19 . The non-transitory storage medium of claim 12 , wherein each of the models receives a subset of features from all features of all sensors and wherein each of the subsets includes one or more features from at least one sensor.
20 . The non-transitory storage medium of claim 19 , wherein the model score is configured to reduce an influence of features that are faulty, noisy, or missing from inferences generated by the selected deployed model.