IP Library › Granted Patent US 12694344

Granted Patent B2

US 12694344 · App. 17/684,602 · Granted Jul 28, 2026

Framework for machine-learning model segmentation

Inventors: Chunming Liu (Seattle, WA); Margaret Elizabeth Whorf (Kirkland, WA)

Assignee: Orcacle International Corporation

G06N20/20 G06F18/214 G06F18/2163 G06F18/217

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12694344
App. No.: 17/684,602
Granted: Jul 28, 2026
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

The present disclosure describes techniques for, in addition to a “global model” that is trained using training data, training one or more “segmented” machine learning (ML) models using subsets or segments of the training data, wherein a segmented ML model is trained using a segment or subset of the training data that satisfies a particular condition. Both the trained global model, and where applicable, one or more trained segmented models are used to make predictions in a production environment.

Claims (50)

1 . A method comprising:

for a segment-defining condition based on one or more features associated with a training dataset, identifying a subset of the training dataset that satisfies the segment-defining condition, wherein a number of training samples included in the subset of the training data set is less than the number of training samples included in the training dataset;

training, in a model training environment, a machine-learning model using the subset of the training dataset to generate a trained segmented machine-learning model;

evaluating a performance of the trained segmented machine-learning model with respect to a trained global machine learning model that is trained based on the training dataset;

deploying the trained segmented machine-learning model and the trained global machine learning model in a production environment responsive to the performance of the trained segmented machine-learning model satisfying a condition; and

responsive to receiving real time data in the production environment, executing the trained segmented machine learning model and the trained global machine learning model to generate a prediction with respect to the real time data, wherein the production environment is configured to transmit the prediction and the real time data to the machine-learning model executed in the training environment.

2 . The method of claim 1 , further comprising:

determining whether the subset of the training dataset satisfies a training criteria; and

responsive to the subset of the training dataset satisfying the training criteria, executing the training of the machine-learning model using the subset of the training dataset to generate the trained segmented machine-learning model.

3 . The method of claim 2 , wherein the training criteria includes a first criterion corresponding to determining whether the number of training samples included in the subset of the training dataset is greater than a first threshold number of training samples.

4 . The method of claim 2 , wherein the training criteria includes a second criterion corresponding to determining whether the number of training samples included in the subset of the training dataset having a particular label is greater than a second threshold number of training samples associated with the particular label.

5 . The method of claim 1 , wherein the performance of the trained segmented machine-learning model and the trained global machine learning model is evaluated based on a metric, the metric being one of an accuracy metric, a recall metric, a precision metric, or an F1-score metric.

6 . The method of claim 1 , wherein the step of evaluating the performance of the trained segmented machine-learning model and the trained global machine learning model further comprises:

determining whether the performance of the trained segmented machine-learning model exceeds the performance of the trained global machine learning model by a predetermined threshold; and

storing the trained segmented machine-learning model in response to a successful determination.

7 . The method of claim 1 , wherein the training the machine-learning model using the subset of the training dataset further comprises:

generating a plurality of machine-learning models based on a set of hyperparameters, wherein each of the plurality of machine-learning models is associated with a unique setting of the set of hyperparameters.

8 . The method of claim 7 , further comprising:

training each machine-learning model of the plurality of machine-learning models using the subset of the training dataset;

evaluating a performance of each machine-learning model of the plurality of machine-learning models based on a metric; and

selecting, based on the evaluating, one of the plurality of machine-learning models to be the trained segmented machine-learning model.

9 . A non-transitory computer readable medium storing specific computer-executable instructions that, when executed by a processor, cause a computer system to perform operations comprising:

training, in a model training environment, a machine-learning model using the subset of the training dataset to generate a trained segmented machine-learning model;

evaluating a performance of the trained segmented machine-learning model with respect to a trained global machine learning model that is trained based on the training dataset;

10 . The non-transitory computer readable medium storing specific computer-executable instructions of claim 9 , further comprising:

determining whether the subset of the training dataset satisfies a training criteria; and

11 . The non-transitory computer readable medium storing specific computer-executable instructions of claim 10 , wherein the training criteria includes a first criterion corresponding to determining whether the number of training samples included in the subset of the training dataset is greater than a first threshold number of training samples.

12 . The non-transitory computer readable medium storing specific computer-executable instructions of claim 10 , wherein the training criteria includes a second criterion corresponding to determining whether the number of training samples included in the subset of the training dataset having a particular label is greater than a second threshold number of training samples associated with the particular label.

13 . The non-transitory computer readable medium storing specific computer-executable instructions of claim 9 , wherein the performance of the trained segmented machine-learning model and the trained global machine learning model is evaluated based on a metric, the metric being one of an accuracy metric, a recall metric, a precision metric, or an F1-score metric.

14 . A computing device comprising:

a processor; and

a memory including instructions that, when executed with the processor, cause the computing device to, at least:

for a segment-defining condition based on one or more features associated with a training dataset, identify a subset of the training dataset that satisfies the segment-defining condition, wherein a number of training samples included in the subset of the training data set is less than the number of training samples included in the training dataset;

train, in a model training environment, a machine-learning model using the subset of the training dataset to generate a trained segmented machine-learning model;

evaluate a performance of the trained segmented machine-learning model with respect to a trained global machine learning model that is trained based on the training dataset;

deploy the trained segmented machine-learning model and the trained global machine learning model in a production environment responsive to the performance of the trained segmented machine-learning model satisfying a condition; and

responsive to receiving real time data in the production environment, execute the trained segmented machine learning model and the trained global machine learning model to generate a prediction with respect to the real time data, wherein the production environment is configured to transmit the prediction and the real time data to the machine-learning model executed in the training environment.

15 . The computing device of claim 14 , further configured to:

determine whether the subset of the training dataset satisfies a training criteria; and

responsive to the subset of the training dataset satisfying the training criteria, train the machine-learning model using the subset of the training dataset to generate the trained segmented machine-learning model.

16 . The computing device of claim 15 , wherein the training criteria includes a first criterion corresponding to determining whether the number of training samples included in the subset of the training dataset is greater than a first threshold number of training samples.

17 . The computing device of claim 15 , wherein the training criteria includes a second criterion corresponding to determining whether the number of training samples included in the subset of the training dataset having a particular label is greater than a second threshold number of training samples associated with the particular label.

18 . The computing device of claim 14 , wherein the performance of the trained segmented machine-learning model and the trained global machine learning model is evaluated based on a metric, the metric being one of an accuracy metric, a recall metric, a precision metric, or an F1-score metric.

19 . The computing device of claim 14 , further configured to:

generate a plurality of machine-learning models based on a set of hyperparameters, wherein each of the plurality of machine-learning models is associated with a unique setting of the set of hyperparameters.

20 . The computing device of claim 14 , wherein the computing device is configured to train the machine-learning model using the subset of the training dataset by generating a plurality of machine-learning models based on a set of hyperparameters, wherein each of the plurality of machine-learning models is associated with a unique setting of the set of hyperparameters.