IP Library Granted Patent US 11,288,602
Granted Patent B2
US 11,288,602 · App. 17/025,889 · Granted Mar 29, 2022

Computer-based systems, computing components and computing objects configured to implement dynamic outlier bias reduction in machine learning models

Inventor: Richard B. Jones (Georgetown, TX)
Assignee: HARTFORD STEAM BOILER INSPECTION AND INSURANCE COMPANY
G06N20/20G06F9/4881G06K9/6256G06K9/6284
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,288,602
App. No.
17/025,889
Granted
Mar 29, 2022
Kind
B2
Abstract

Systems and methods include processors for receiving training data for a user activity; receiving bias criteria; determining a set of model parameters for a machine learning model including: (1) applying the machine learning model to the training data; (2) generating model prediction errors; (3) generating a data selection vector to identify non-outlier target variables based on the model prediction errors; (4) utilizing the data selection vector to generate a non-outlier data set; (5) determining updated model parameters based on the non-outlier data set; and (6) repeating steps (1)-(5) until a censoring performance termination criterion is satisfied; training classifier model parameters for an outlier classifier machine learning model; applying the outlier classifier machine learning model to activity-related data to determine non-outlier activity-related data; and applying the machine learning model to the non-outlier activity-related data to predict future activity-related attributes for the user activity.

Claims (101)

1. A method comprising:

receiving, by at least one processor, a training data set of target variables representing at least one activity-related attribute for at least one user activity;

receiving, by the at least one processor, at least one bias criteria used to determine one or more outliers;

determining, by the at least one processor, a data selection vector to select non-outlier data elements of the training data set comprising:

(1) applying, by the at least one processor, a reference machine learning model having a set of initial model parameters to the training data set to determine a set of model predicted values;

(2) generating, by the at least one processor, an error set of data element errors by comparing the set of model predicted values to corresponding actual values of the training data set;

(3) generating, by the at least one processor, the data selection vector to identify non-outlier target variables based at least in part on the error set of data element errors and the at least one bias criteria;

(4) utilizing, by the at least one processor, the data selection vector on the training data set to generate a non-outlier data set;

(5) determining, by the at least one processor, a set of updated model parameters for the reference machine learning model based on the non-outlier data set; and

(6) repeating, by the at least one processor, steps (1)-(5) as an iteration until at least one censoring performance termination criterion is satisfied so as to obtain the set of model parameters for the reference machine learning model as the updated model parameters, whereby each iteration re-generates the set of predicted values, the error set, the data selection vector, and the non-outlier data set using the set of updated model parameters as the set of initial model parameters to measure improvements in accuracy based on the error set of each iteration;

generating, by the at least one processor, an outlier data set and an non-outlier dataset based at least in part on the data selection vector;

training, by the at least one processor, based at least in part on the outlier data set, a set of classifier model parameters of a classifier machine learning model to obtain a trained classifier machine learning model trained to classify outlier data elements;

training, by the at least one processor, based at least in part on the non-outlier data set, a set of non-outlier model parameters of a base machine learning model to obtain a trained base machine learning model trained to predict values of non-outlier data elements; and

outputting, by the at least one processor, the trained classifier machine learning model and the trained base machine learning model.

2. The method of claim 1 , further comprising:

applying, by the at least one processor, the data selection vector to the training data set to determine an outlier training data set;

training, by the at least one processor, using the outlier training data set, at least one outlier-specific model parameter of at least one outlier-specific machine learning model to predict outlier data values; and

utilizing, by the at least one processor, the outlier-specific machine learning model to predict outlier activity-related data values for the set of outlier activity-related data.

3. The method of claim 1 , further comprising:

training, by the at least one processor, using the training data set, generalized model parameters of a generalized machine learning model to predict data values;

utilizing, by the at least one processor, the generalized machine learning model to predict outlier activity-related data values for the set of outlier activity-related data; and

utilizing, by the at least one processor, the generalized machine learning model to predict the activity-related data values.

4. The method of claim 1 , further comprising:

applying, by the at least one processor, the data selection vector to the training data set to determine an outlier training data set;

training, by the at least one processor, using the outlier training data set, an outlier-specific model parameters of an outlier-specific machine learning model to predict outlier data values;

training, by the at least one processor, using the training data set, generalized model parameters of a generalized machine learning model to predict data values;

utilizing, by the at least one processor, the outlier-specific machine learning model to predict outlier activity-related data values for the set of outlier activity-related data; and

utilizing, by the at least one processor, the outlier-specific machine learning model to predict the activity-related data values.

5. The method of claim 1 , further comprising:

training, by the at least one processor, using the training data set, generalized model parameters of a generalized machine learning model to predict data values;

utilizing, by the at least one processor, the generalized machine learning model to predict the activity-related data values for the set of activity-related data;

utilizing, by the at least one processor, the outlier classifier machine learning model to identify outlier activity-related data values of the activity-related data values; and

removing, by the at least one processor, the outlier activity-related data values.

6. The method of claim 1 , wherein the training data set comprises the at least one activity-related attribute of concrete compressive strength as a function of concrete composition and concrete curing exposure.

7. The method of claim 1 , wherein the training data set comprises the at least one activity-related attribute of energy use data as a function of household environmental conditions and lighting conditions.

8. The method of claim 1 , further comprising:

receiving, by the at least one processor, an application programming interface (API) request to generate a prediction with at least one data element; and

instantiating, by the at least one processor, at least one cloud computing resource to schedule execution of the machine learning model;

utilizing, by the at least one processor according to the schedule for execution, the machine learning model to predict at least one activity-related data element value for the at least one data element; and

returning, by the at least one processor, the at least one activity-related data element value to a computing device associated with the API request.

9. The method of claim 1 , wherein the training data set comprises the at least one activity-related attribute of three-dimensional patient imagery of a medical dataset; and

wherein the machine learning model is configured to predict the activity-related data values comprising two or more physically-based rendering parameters based on the medical dataset.

10. The method of claim 1 , wherein the training data set comprises the at least one activity-related attribute of simulated control results for electronic machine commands; and

wherein the machine learning model is configured to predict the activity-related data values comprising control commands for the electronic machine.

11. The method of claim 1 , further comprising:

splitting, by the at least one processor, the set of activity-related data into a plurality of subsets of activity-related data;

determining, by the at least one processor, an ensemble model for each subset of activity-related data of the plurality of subsets of activity-related data;

wherein the machine learning model comprises an ensemble of models;

wherein each ensemble model comprises a random combination of models from the ensemble of models;

utilizing, by the at least one processor, each ensemble model separately to predict ensemble-specific activity-related data values;

determining, by the at least one processor, an error for each ensemble model based on the ensemble-specific activity-related data values and known values; and

selecting, by the at least one processor, a highest performing ensemble model based on a lowest error.

12. A system comprising:

at least one processor in communication with a non-transitory computer-readable storage medium having software instructions stored thereon, wherein the software instructions, when executed, cause the at least one processor to perform steps to:

receive a training data set of target variables representing at least one activity-related attribute for at least one user activity;

receive at least one bias criteria used to determine one or more outliers;

determine data selection vector to select non-outlier data elements of the training data set comprising:

(1) apply the reference machine learning model having a set of initial model parameters to the training data set to determine a set of model predicted values;

(2) generate an error set of data element errors by comparing the set of model predicted values to corresponding actual values of the training data set;

(3) generate the data selection vector to identify non-outlier target variables based at least in part on the error set of data element errors and the at least one bias criteria;

(4) utilize the data selection vector on the training data set to generate a non-outlier data set;

(5) determine a set of updated model parameters for the reference machine learning model based on the non-outlier data set; and

(6) repeat steps (1)-(5) as an iteration until at least one censoring performance termination criterion is satisfied so as to obtain the set of model parameters for the reference machine learning model as the updated model parameters, whereby each iteration re-generates the set of predicted values, the error set, the data selection vector, and the non-outlier data set using the set of updated model parameters as the set of initial model parameters to measure improvements in accuracy based on the error set of each iteration;

generate an outlier data set and an non-outlier dataset based at least in part on the data selection vector;

train based at least in part on the outlier data set, a set of classifier model parameters of a classifier machine learning model to obtain a trained classifier machine learning model trained to classify outlier data elements;

train based at least in part on the non-outlier data set, a set of non-outlier model parameters of a base machine learning model to obtain a trained base machine learning model trained to predict values of non-outlier data elements; and

output the trained classifier machine learning model and the trained base machine learning model.

13. The system of claim 12 , wherein the software instructions, when executed, further cause the at least one processor to perform steps to:

apply the data selection vector to the training data set to determine an outlier training data set;

train using the outlier training data set, at least one outlier-specific model parameter of at least one outlier-specific machine learning model to predict outlier data values; and

utilize the outlier-specific machine learning model to predict outlier activity-related data values for the set of outlier activity-related data.

14. The system of claim 12 , wherein the software instructions, when executed, further cause the at least one processor to perform steps to:

train using the training data set, generalized model parameters of a generalized machine learning model to predict data values;

utilize the generalized machine learning model to predict outlier activity-related data values for the set of outlier activity-related data; and

utilize generalized machine learning model to predict the activity-related data values.

15. The system of claim 12 , wherein the software instructions, when executed, further cause the at least one processor to perform steps to:

apply the data selection vector to the training data set to determine an outlier training data set;

train using the outlier training data set, an outlier-specific model parameters of an outlier-specific machine learning model to predict outlier data values;

train using the training data set, generalized model parameters of a generalized machine learning model to predict data values;

utilize the outlier-specific machine learning model to predict outlier activity-related data values for the set of outlier activity-related data; and

utilize the outlier-specific machine learning model to predict the activity-related data values.

16. The system of claim 12 , wherein the software instructions, when executed, further cause the at least one processor to perform steps to:

train using the training data set, generalized model parameters of a generalized machine learning model to predict data values;

utilize the generalized machine learning model to predict the activity-related data values for the set of activity-related data;

utilize the outlier classifier machine learning model to identify outlier activity-related data values of the activity-related data values; and

remove the outlier activity-related data values.

17. The system of claim 12 , wherein the training data set comprises the at least one activity-related attribute of concrete compressive strength as a function of concrete composition and concrete curing exposure.

18. The system of claim 12 , wherein the training data set comprises the at least one activity-related attribute of energy use data as a function of household environmental conditions and lighting conditions.

19. The system of claim 12 , wherein the software instructions, when executed, further cause the at least one processor to perform steps to:

receive an application programming interface (API) request to generate a prediction with at least one data element; and

instantiate at least one cloud computing resource to schedule execution of the machine learning model;

utilize, according to the schedule for execution, the machine learning model to predict at least one activity-related data element value for the at least one data element; and

return the at least one activity-related data element value to a computing device associated with the API request.

20. The system of claim 12 , wherein the software instructions, when executed, further cause the at least one processor to perform steps to:

split the set of activity-related data into a plurality of subsets of activity-related data;

determine an ensemble model for each subset of activity-related data of the plurality of subsets of activity-related data;

wherein the machine learning model comprises an ensemble of models;

wherein each ensemble model comprises a random combination of models from the ensemble of models;

utilize each ensemble model separately to predict ensemble-specific activity-related data values;

determine an error for each ensemble model based on the ensemble-specific activity-related data values and known values; and

select a highest performing ensemble model based on a lowest error.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY NAME PREVIOUSLY RECORDED AT REEL: 53822 FRAME: 931. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Apr 30, 2026
From: JONES, RICHARD B.
To: THE HARTFORD STEAM BOILER INSPECTION AND INSURANCE COMPANY
Reel/Frame 075371/0340 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 18, 2020
From: JONES, RICHARD B.
To: HARTFORD STEAM BOILER INSPECTION AND INSURANCE COMPANY
Reel/Frame 053822/0931 →
Continuity (2)
Provisional Application 62902074 · Sep 18, 2019
Related Publication 20210110313A1 · Apr 15, 2021
Cited By (4)
US 12,353,506 US 12,613,231 US 12,626,152 US 12,653,402