IP Library Patent Application 18404159
Patent Application
App. No. 18/404,159

TECHNIQUES FOR TRAINING AND ANALYZING A MACHINE LEARNING MODEL

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
18/404,159
Abstract

Disclosed techniques relate to improving predictions of machine learning models. In an example, a method involves receiving, from a machine learning model, predictions associated with a class of scenarios. The method includes identifying, in response to receiving the predictions, a subset of the class of scenarios that are beyond a threshold tolerance of accuracy. The method includes based on identifying the subset of the class of scenarios a training data set that includes emphasized event data from a plurality of historical sporting events. The method includes generating an updated machine learning model and deploying the updated machine learning model.

Claims (82)

1 . A method comprising:

receiving, from a machine learning model, predictions associated with a class of scenarios;

identifying, in response to receiving the predictions, a subset of the class of scenarios that are beyond a threshold tolerance of accuracy;

based on identifying the subset of the class of scenarios, generating, by a computing system, a training data set that includes emphasized event data from a plurality of historical sporting events, wherein the emphasized event data emphasizes events associated with the subset of the class of scenarios;

generating, by the computing system, an updated machine learning model by:

identifying weights of the machine learning model,

initializing the updated machine learning model using the weights, and

training the updated machine learning model using the training data set, wherein training the updated machine learning model comprises modifying the weights based on the training data set; and

deploying, by the computing system, the updated machine learning model.

2 . The method of claim 1 , further comprising:

receiving updated predictions, associated with the class of scenarios, from the updated machine learning model; and

outputting a visualization of the updated predictions.

3 . The method of claim 2 , further comprising:

identifying, from the updated predictions, an additional subset of the class of scenarios that are beyond the threshold tolerance of accuracy; and

based on identifying the additional subset of the class of scenarios, generating, by the computing system, an additional training data set that includes emphasized additional event data from the plurality of historical sporting events, wherein the emphasized additional event data emphasizes the additional subset of the class of scenarios.

4 . The method of claim 3 , wherein training the updated machine learning model using the training data set comprising:

sequentially exposing the updated machine learning model to boosts of training data comprising the training data set and the additional training data set.

5 . The method of claim 1 , wherein identifying the class of scenarios comprises:

receiving an indication from a user that the machine learning model is generating predictions beyond the threshold tolerance of accuracy.

6 . The method of claim 1 , wherein identifying the class of scenarios comprises:

providing a first set of inputs to the machine learning model to generate a first prediction;

providing a second set of inputs to the machine learning model to generate an additional prediction;

determining that the first prediction is within an expected range of a first expected prediction; and

determining the second prediction is outside of a second expected range of an second expected prediction.

7 . The method of claim 1 , wherein the emphasized event data is generated by:

providing initial training data used to train the machine learning model to an edge event machine learning model;

providing the subset of the class of scenarios to the edge event machine learning model; and

receiving the emphasized event data output by the edge event machine learning model based on the initial training data and the subset of the class of scenarios, wherein the emphasized event data is a subset of the initial training data weighted higher than in the initial training data.

8 . A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:

receiving, from a machine learning model, predictions associated with a class of scenarios;

identifying, in response to receiving the predictions, a subset of the class of scenarios that are beyond a threshold tolerance of accuracy;

based on identifying the subset of the class of scenarios, generating, by a computing system, a training data set that includes emphasized event data from a plurality of historical sporting events, wherein the emphasized event data emphasizes events associated with the subset of the class of scenarios;

generating, by the computing system, an updated machine learning model by:

identifying weights of the machine learning model,

initializing the updated machine learning model using the weights, and

training the updated machine learning model using the training data set, wherein training the updated machine learning model comprises modifying the weights based on the training data set; and

deploying, by the computing system, the updated machine learning model.

9 . The non-transitory computer readable medium of claim 8 , wherein when executed by a processor, the instructions perform operations comprising:

receiving updated predictions, associated with the class of scenarios, from the updated machine learning model; and

outputting a visualization of the updated predictions.

10 . The non-transitory computer readable medium of claim 9 , wherein when executed by a processor, the instructions perform operations comprising:

identifying, from the updated predictions, an additional subset of the class of scenarios that are beyond the threshold tolerance of accuracy; and

based on identifying the additional subset of the class of scenarios, generating, by the computing system, an additional training data set that includes emphasized additional event data from the plurality of historical sporting events, wherein the emphasized additional event data emphasizes the additional subset of the class of scenarios.

11 . The non-transitory computer readable medium of claim 10 , wherein training the updated machine learning model using the training data set comprising:

sequentially exposing the updated machine learning model to boosts of training data comprising the training data set and the additional training data set.

12 . The non-transitory computer readable medium of claim 10 , wherein identifying the class of scenarios comprises:

receiving an indication from a user that the machine learning model is generating predictions beyond the threshold tolerance of accuracy.

13 . The non-transitory computer readable medium of claim 10 , wherein identifying the class of scenarios comprises:

providing a first set of inputs to the machine learning model to generate a first prediction;

providing a second set of inputs to the machine learning model to generate a second prediction;

determining that the first prediction is within an expected range of a first expected prediction; and

determining the second prediction is outside of a second expected range of an second expected prediction.

14 . The non-transitory computer readable medium of claim 10 , wherein the emphasized event data is generated by:

providing initial training data used to train the machine learning model to an edge event machine learning model;

providing the subset of the class of scenarios to the edge event machine learning model; and

receiving the emphasized event data output by the edge event machine learning model based on the initial training data and the subset of the class of scenarios, wherein the emphasized event data is a subset of the initial training data weighted higher than in the initial training data.

15 . A system comprising:

a processor; and

a non-transitory computer readable medium having program instructions stored thereon, which, when executed by the processor, cause the system to perform operations comprising:

receiving, from a machine learning model, predictions associated with a class of scenarios;

identifying, in response to receiving the predictions, a subset of the class of scenarios that are beyond a threshold tolerance of accuracy;

based on identifying the subset of the class of scenarios, generating, by a computing system, a training data set that includes emphasized event data from a plurality of historical sporting events, wherein the emphasized event data emphasizes events associated with the subset of the class of scenarios;

generating, by the computing system, an updated machine learning model by:

identifying weights of the machine learning model,

initializing the updated machine learning model using the weights, and

training the updated machine learning model using the training data set, wherein training the updated machine learning model comprises modifying the weights based on the training data set; and

deploying, by the computing system, the updated machine learning model.

16 . The system of claim 15 , wherein when executed by the processor, the program instructions cause the system to perform operations comprising:

receiving updated predictions, associated with the class of scenarios, from the updated machine learning model; and

outputting a visualization of the updated predictions.

17 . The system of claim 16 , wherein when executed by the processor, the program instructions cause the system to perform operations comprising:

identifying, from the updated predictions, an additional subset of the class of scenarios that are beyond the threshold tolerance of accuracy; and

based on identifying the additional subset of the class of scenarios, generating, by the computing system, an additional training data set that includes emphasized additional event data from the plurality of historical sporting events, wherein the emphasized additional event data emphasizes the additional subset of the class of scenarios.

18 . The system of claim 17 , wherein training the updated machine learning model using the training data set comprising:

sequentially exposing the updated machine learning model to boosts of training data comprising the training data set and the additional training data set.

19 . The system of claim 16 , wherein identifying the class of scenarios comprises:

receiving an indication from a user that the machine learning model is generating predictions beyond the threshold tolerance of accuracy.

20 . The system of claim 16 , wherein identifying the class of scenarios comprises:

providing a first set of inputs to the machine learning model to generate a first prediction;

providing a second set of inputs to the machine learning model to generate a second prediction;

determining that the first prediction is within an expected range of a first expected prediction; and

determining the second prediction is outside of a second expected range of a second expected prediction.

Assignments (2)
SECURITY INTEREST Recorded Apr 14, 2026
From: STATS LLC
To: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
Reel/Frame 075390/0491 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 12, 2024
From: OLIVER, DOMINIC; STÖCKL, MICHAEL; MURALI, ARUN; LUCEY, PATRICK JOSEPH
To: STATS LLC
Reel/Frame 066723/0821 →