IP Library Granted Patent US 11,416,751
Granted Patent B2
US 11,416,751 · App. 15/937,648 · Granted Aug 16, 2022

Time-based ensemble machine learning model

Inventors: SriSatish Ambati (Palo Alto, CA); Ashrith Barthur (San Francisco, CA)
Assignee: H2O.ai Inc.
G06N5/02G06N20/00G06N20/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,416,751
App. No.
15/937,648
Granted
Aug 16, 2022
Kind
B2
Abstract

An input dataset is sorted into a first version of data and a second version of data. The first version of data is associated with a first period of time and the second version of data is associated with a second period of time. The second period of time is a shorter period of time than the first period of time. A first set of one or more machine learning models is generated based on the first version of data. A second set of one or more machine learning models is generated based on the second version of data. The first set of one or more machine learning models and the second set of one or more machine learning models are combined to generate an ensemble model. A prediction based on the ensemble model is outputted. The prediction indicates abnormal behavior associated with the input dataset.

Claims (75)

1. A system, comprising:

a processor configured to:

receive an input dataset, wherein the input dataset is comprised of a plurality of entries, and wherein the plurality of entries are associated with a plurality of features and corresponding feature values;

sort the input dataset into a first version of data and a second version of data, wherein the first version of data is associated with a first period of time and the second version of data is associated with a second period of time, wherein the second period of time is a shorter period of time than the first period of time;

generate a first set of one or more machine learning models based on the first version of data;

generate a second set of one or more machine learning models based on the second version of data;

combine the first set of one or more machine learning models and the second set of one or more machine learning models to generate a combined ensemble model;

select features included in the input dataset, comprising to:

determine a distribution of feature values associated with at least one feature included in the input dataset;

determine whether the distribution satisfied a threshold variation; and

in response to a determination that the distribution does not satisfy the threshold variation, omit selecting the at least one feature;

train a machine learning model based at least in part on the selected features of the input dataset;

output a predication based on the combined ensemble model, wherein the prediction indicates abnormal behavior associated with the input dataset;

determine a baseline value for a feature of the plurality of features, wherein the baseline value for the feature corresponds with a first statistical value of the feature for the first period of time;

determine a volatility value for the feature, wherein the volatility value for the feature corresponds with a second statistical value of the feature for the second period of time;

compare the baseline value with the volatility value; and

in response to a determination that the baseline value varies from the volatility value more than a threshold, retrain at least one machine learning model, comprising to:

retrain the at least one machine learning model based on the volatile feature to tune a weight associated with the volatile feature; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1 , wherein the processor is further configured to generate one or more new features based at least in part on the plurality of features.

3. The system of claim 2 , wherein the processor is further configured to:

generate a machine learning model based at least in part on the one or more new features.

4. The system of claim 1 , wherein the combined ensemble model is configured to output one or more predictions.

5. The system of claim 1 , wherein the processor is further configured to apply production data to the first set of one or more machine learning models and the second set of one or more machine learning models.

6. The system of claim 1 , wherein the processor is further configured to:

receive an indication of one or more false positives; and

retrain a machine learning model based at least in part on the indication.

7. The system of claim 1 , wherein the processor is further configured to:

sample a subset of one or more predictions associated with the combined ensemble model;

determine whether a trigger event is satisfied; and

retrain a machine learning model based on the determination.

8. The system of claim 1 , wherein the first set of one or more machine learning models and/or the second set of one or more machine learning models are periodically trained.

9. A method, comprising:

receiving an input dataset, wherein the input dataset is comprised of a plurality of entries, and wherein the plurality of entries are associated with a plurality of features and corresponding feature values;

sorting the input dataset into a first version of data and a second version of data, wherein the first version of data is associated with a first period of time and the second version of data is associated with a second period of time, wherein the second period of time is a shorter period of time than the first period of time;

generating a first set of one or more machine learning models based on the first version of data;

generating a second set of one or more machine learning models based on the second version of data;

combining the first set of one or more machine learning models and the second set of one or more machine learning models to generate a combined ensemble model;

selecting features included in the input dataset, comprising:

determining a distribution of feature values associated with at least one feature included in the input dataset;

determining whether the distribution satisfied a threshold variation; and

in response to a determination that the distribution does not satisfy the threshold variation, omitting to select the at least one feature;

training a machine learning model based at least in part on the selected features of the input dataset;

output a prediction based on the combined ensemble model, wherein the prediction indicates abnormal behavior associated with the input dataset;

determining a baseline value for a feature of the plurality of features, wherein the baseline value for the feature corresponds with a first statistical value of the feature for the first period of time;

determining a volatility value for the feature, wherein the volatility value for the feature corresponds with a second statistical value of the feature for the second period of time;

comparing the baseline value with the volatility value; and

in response to a determination that the baseline value varies from the volatility value more than a threshold, retraining at least one machine learning model, comprising:

retraining the at least one machine learning model based on the volatile feature to tune a weight associated with the volatile feature.

10. The method of claim 9 , further comprising:

receiving an indication of one or more false positives; and

retraining a machine learning model based at least in part on the indication.

11. The method of claim 9 , further comprising:

sampling a subset of one or more predictions;

determining whether a trigger event is satisfied; and

retraining a machine learning model based on the determination.

12. The method of claim 9 , further comprising receiving the input dataset, wherein the input data is comprised of a plurality of entries, wherein the plurality of entries are associated with a plurality of features and corresponding feature values.

13. The method of claim 12 , further comprising generating one or more new features based at least in part on the plurality of features.

14. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving an input dataset, wherein the input dataset is comprised of a plurality of entries, and wherein the plurality of entries are associated with a plurality of features and corresponding feature values;

sorting the input dataset into a first version of data and a second version of data, wherein the first version of data is associated with a first period of time and the second version of data is associated with a second period of time, wherein the second period of time is a shorter period of time than the first period of time;

generating a first set of one or more machine learning models based on the first version of data;

generating a second set of one or more machine learning models based on the second version of data;

combining the first set of one or more machine learning models and the second set of one or more machine learning models to generate an ensemble model;

selecting features included in the input dataset, comprising:

determining a distribution of feature values associated with at least one feature included in the input dataset;

determining whether the distribution satisfied a threshold variation; and

in response to a determination that the distribution does not satisfy the threshold variation, omitting to select the at least one feature;

training a machine learning model based at least in part on the selected features of the input dataset;

output a prediction based on the ensemble model, wherein the prediction indicates indicate abnormal behavior associated with the input dataset;

determining a baseline value for a feature of the plurality of features, wherein the baseline value for the feature corresponds with a first statistical value of the feature for the first period of time;

determining a volatility value for the feature, wherein the volatility value for the feature corresponds with a second statistical value of the feature for the second period of time;

comparing the baseline value with the volatility value; and

in response to a determination that the baseline value varies from the volatility value more than a threshold, retraining at least one machine learning model, comprising:

retraining the at least one machine learning model based on the volatile feature to tune a weight associated with the volatile feature.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 22, 2018
From: AMBATI, SRISATISH; BARTHUR, ASHRITH
To: H2O.AI INC.
Reel/Frame 046180/0526 →
Continuity (2)
Provisional Application 62479963 · Mar 31, 2017
Related Publication 20180293501A1 · Oct 11, 2018