IP Library › Granted Patent US 11,164,081
Granted Patent B2
US 11,164,081 · App. 16/162,639 · Granted Nov 2, 2021

Synthesizing a singular ensemble machine learning model from an ensemble of models

Inventors: Vincent Pham (Champaign, IL); Mark Watson (Urbana, IL); Jeremy Goodsitt (Champaign, IL); Reza Farivar (Champaign, IL); Austin Walters (Savoy, IL); Kenneth Taylor (Champaign, IL); Fardin Abdi Taghi Abad (Champaign, IL); Anh Truong (Champaign, IL)
Assignee: Capital One Services, LLC
G06N3/08G06N3/04G06N3/0427G06N3/0445G06N3/0454
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,164,081
App. No.
16/162,639
Granted
Nov 2, 2021
Kind
B2
Abstract

The present disclosure relates to systems and methods for generating and using a singular ensemble model.

Claims (90)

1. A system for generating a singular ensemble model, comprising:

a model generator configured to:

obtain information from at least one of a database, a client device, or an input recognizer; and

generate a plurality of machine learning models based on the obtained information, the models having a plurality of layers, a plurality of nodes, and a model type;

a model clusterer configured to:

obtain the models from the model generator; and

automatically generate clusters of the models, the clusters comprising a plurality of the models having at least one of a similar input type or a similar output type;

at least one processor; and

at least one storage medium storing instructions that, when executed, configure the processor to perform operations comprising:

obtaining, from the model clusterer, a plurality of the clustered models;

obtaining a training data set consistent with the clustered models from at least one of the database, the client device, or the input recognizer;

applying the clustered models to the training data set to obtain outputs associated with the clustered models;

mapping the outputs to features of the clustered models;

combining the mapped features into a singular machine learning model;

training the singular machine learning model using the training data set; and

outputting the trained singular machine learning model, the outputting comprising transmitting the trained singular machine learning model to a client device, the client device being configured to:

store a data structure defining parameters of the trained singular machine learning model;

retrieve input data for the trained singular machine learning model;

apply the trained singular machine learning model to the input data to generate results; and

display the results.

2. The system of claim 1 , wherein the models comprise at least one neural network.

3. The system of claim 1 , wherein the models comprise at least one linear regression.

4. The system of claim 1 , wherein the operations further comprise:

determining whether the models comprise a same model type;

when the models are determined to comprise the same model type, selecting the same model type for the singular machine learning model; and

when the models are determined to comprise different model types, selecting a neural network type for the singular machine learning model.

5. The system of claim 1 , wherein the singular machine learning model comprises a neural network that overfits the clustered models.

6. The system of claim 5 , wherein:

the singular machine learning model comprises a plurality of layers; and

a number of the singular machine learning model layers is at least as great as a number of layers in a first one of the models having a largest number of layers.

7. The system of claim 6 , wherein each layer of the singular machine learning model comprises a plurality of nodes, the number of the singular machine learning model nodes being at least as great as the number of nodes in corresponding layers in one of the models having a largest number of nodes.

8. The system of claim 1 , wherein mapping the outputs to features of the models comprises applying one or more weights to the outputs during mapping.

9. The system of claim 8 , wherein the one or more weights are equal to each other.

10. The system of claim 8 , wherein the one or more weights comprise inputs from a user.

11. The system of claim 1 , wherein outputting the trained singular machine learning model further comprises at least one of storing the trained singular machine learning model in the at least one storage medium.

12. A system for generating a singular ensemble model, comprising:

a model generator configured to:

obtain information from at least one of a database, a client device, or an input recognizer; and

generate a plurality of machine learning models based on the obtained information and consistent with a requested prediction included in the obtained, the models having a plurality of layers, a plurality of nodes, and a model type, wherein the requested prediction predicts input of a user based on features extracted from partial inputs;

a model clusterer configured to:

obtain the models from the model generator; and

automatically generate clusters of the models, the clusters comprising a plurality of the models having at least one of a similar input type or similar output type;

at least one processor; and

at least one storage medium storing instructions that, when executed, configure the processor to perform operations comprising:

obtaining, from the model clusterer, a plurality of the clustered models;

obtaining a plurality of training data sets consistent with the plurality of clustered models from at least one of the database, the client device, or the input recognizer, each set corresponding to one of the models;

applying the clustered models to the corresponding training data sets to obtain output sets associated with the models, each output set corresponding to one of the clustered models,

combining the output sets to form a final output set,

mapping the final output set to features of the clustered models;

combining the mapped features of the models into a singular machine learning model;

training the singular machine learning model using the corresponding training data sets; and

outputting the trained singular machine learning model, the outputting comprising transmitting the trained singular machine learning model to a client device, the client device being configured to:

store a data structure defining parameters of the trained singular machine learning model;

retrieve input data for the trained singular machine learning model;

apply the trained singular machine learning model to the input data to generate results; and

display the results.

13. The system of claim 12 , wherein combining the output sets to form a final output set comprises applying one or more weights to the outputs during mapping.

14. The system of claim 13 , wherein the one or more weights are equal to each other.

15. The system of claim 13 , wherein the one or more weights comprise inputs from a user.

16. The system of claim 12 , wherein the mapped features of the models comprise feature vectors extracted from the training data sets.

17. The system of claim 12 , wherein the operations further comprise training the singular machine learning model using at least one new training data set.

18. The system of claim 12 , wherein training the singular machine learning model comprises recursive adjustments of one or more parameters of the singular machine learning model.

19. The system of claim 18 , wherein the recursive adjustments are configured to reduce at least one of root-mean-square deviation (RMSD), Akaike information criterion (AIC), or logarithmic loss (LOGLOSS).

20. A system for generating a singular ensemble model, comprising:

an input recognizer configured to:

receive a request for a model from a client device;

provide, based on the request, an indication to a model generator to generate a model; and

cluster data to create a training data set;

a model generator configured to:

obtain information from the input recognizer; and

generate a plurality of machine learning models based on the obtained information, the models having a plurality of layers, a plurality of nodes, and a model type;

a model clusterer configured to:

obtain the models from the model generator; and

automatically generate clusters of the models, the clusters comprising a plurality of the models having at least one of a similar input type or similar output type;

at least one processor; and

at least one storage medium storing instructions that, when executed, configure the processor to perform operations comprising:

obtaining, from the model clusterer, a plurality of the clustered models;

obtaining, from the input recognizer, the training data set, the training data set being consistent with the plurality of clustered models;

applying the plurality of clustered models to the training data set to obtain outputs associated with the models;

mapping the outputs to features of the models;

combining the mapped features of the models into a singular machine learning model;

applying the singular machine learning model to the training data set to obtain output;

comparing the outputs to the output of the singular machine learning model;

adjusting one or more parameters of the singular machine learning model based on the comparison;

determining whether the comparing and the adjusting comply with at least one threshold, wherein the threshold comprises at least one of a direct threshold or a threshold relative to a last iteration of parameter adjustment; and

outputting the trained singular machine learning model to the client device, the client device being configured to:

store a data structure defining parameters of the trained singular machine learning model;

retrieve input data for the trained singular machine learning model;

apply the trained singular machine learning model to the input data to generate results; and

display the results.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 17, 2018
From: PHAM, VINCENT; WATSON, MARK; GOODSITT, JEREMY; FARIVAR, REZA; WALTERS, AUSTIN; TAYLOR, KENNETH; ABDI TAGHI ABAD, FARDIN; TRUONG, ANH
To: CAPITAL ONE SERVICES, LLC
Reel/Frame 047195/0945 →
Continuity (1)
Related Publication 20200125942A1 · Apr 23, 2020