IP Library Granted Patent US 12,354,006
Granted Patent B2
US 12,354,006 · App. 18/405,218 · Granted Jul 8, 2025

System and method for multi-task learning

Inventors: Matthew Holbrook (Chicago, IL); Jennifer Hobbs (Chicago, IL); Patrick Joseph Lucey (Chicago, IL)
Assignee: STATS LLC
G06N3/08G06N3/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,354,006
App. No.
18/405,218
Granted
Jul 8, 2025
Kind
B2
Abstract

A method of generating a multi-modal prediction is disclosed herein. A computing system retrieves event data from a data store. The event data includes information for a plurality of events across a plurality of seasons. Computing system generates a predictive model using a mixture density network, by generating an input vector from the event data learning, by the mixture density network, a plurality of values associated with a next play following each play in the event data. The mixture density network is trained to output the plurality of values near simultaneously. Computing system receives a set of event data directed to an event in a match. The set of event data includes information directed to at least playing surface position and current score. Computing system generates, via the predictive model, a plurality of values associated with a next event following the event based on the set of event data.

Claims (60)

1. A computer-implemented method for generating a multi-modal prediction, the computer-implemented method comprising:

receiving, by one or more processors, match data for a sporting match;

extracting, by the one or more processors, a plurality of parameters associated with an event of a play-by-play event sequence from the match data;

generating, by the one or more processors, an input data set from the plurality of extracted parameters;

generating, by the one or more processors, a multi-modal prediction based on the input data set; and

generating, by the one or more processors, one or more graphical representations of the multi-modal prediction for the event of the play-by-play event sequence.

2. The computer-implemented method of claim 1 , wherein the plurality of parameters include a playing surface position, a subsequent play-by-play event sequence, a plurality of players, at least one team, a team in possession of a ball, or a game context.

3. The computer-implemented method of claim 2 , wherein generating, by the one or more processors, the multi-modal prediction based on the input data set comprises:

providing, by the one or more processors, the plurality of extracted parameters to a mixture density network.

4. The computer-implemented method of claim 1 , wherein generating, by the one or more processors, the multi-modal prediction based on the input data set includes generating predications for at least one of: one or more expected meters, an expected try tackle, an expected try set, a win probability, an expected play selection, or a final score line.

5. The computer-implemented method of claim 1 , the computer-implemented method further comprising:

receiving, by the one or more processors, the match data from one or more tracking systems.

6. The computer-implemented method of claim 1 , wherein generating the input data set includes:

transforming, by the one or more processors, the match data into one or more segmented data sets;

selecting, by the one or more processors, a subset of one or more segmented data sets; and

creating, by the one or more processors, a dense representation of the subset of one or more segmented data sets.

7. The computer-implemented method of claim 6 , wherein generating the input data set further comprises:

providing, by the one or more processors, each of the one or more segmented data sets to one or more embedding layers;

receiving, by the one or more processors, dense output from the one or more embedding layers; and

generating, by the one or more processors, the input data set by concatenating the dense output, one or more continuous features, and spatial information.

8. The computer-implemented method of claim 7 , wherein the one or more continuous features include a score difference, a remaining time, or a playing surface position.

9. A non-transitory computer readable medium comprising one or more sequences of instructions, that, when executed by one or more processors, causes a computing system to perform operations comprising:

receiving, by the computing system, match data for a sporting match;

extracting, by the computing system, a plurality of parameters associated with an event of a play-by-play event sequence from the match data;

generating, by the computing system, an input data set from the plurality of extracted parameters;

generating, by the computing system, a multi-modal prediction based on the input data set; and

generating, by the computing system, one or more graphical representations of the multi-modal prediction for the event of the play-by-play event sequence.

10. The non-transitory computer readable medium of claim 9 , wherein the plurality of parameters include a playing surface position, a subsequent play-by-play event sequence, a plurality of players, at least one team, a team in possession of a ball, or a game context.

11. The non-transitory computer readable medium of claim 10 , wherein generating, by the computing system, the multi-modal prediction based on the input data set comprises:

providing, by the one or more processors, the plurality of extracted parameters to a mixture density network.

12. The non-transitory computer readable medium of claim 9 , wherein generating, by the computing system, the multi-modal prediction based on the input data set includes generating predications for at least one of: one or more expected meters, an expected try tackle, an expected try set, a win probability, an expected play selection, or a final score line.

13. The non-transitory computer readable medium of claim 9 , the operations further comprising:

receiving, by the computing system, the match data from one or more tracking systems.

14. The non-transitory computer readable medium of claim 9 , wherein generating the input data set includes:

transforming, by the computing system, the match data into one or more segmented data sets;

selecting, by the computing system, a subset of one or more segmented data sets; and

creating, by the computing system, a dense representation of the subset of one or more segmented data sets.

15. The non-transitory computer readable medium of claim 14 , wherein generating the input data set further comprises:

providing, by the computing system, each of the one or more segmented data sets to one or more embedding layers;

receiving, by the computing system, dense output from the one or more embedding layers; and

generating, by the computing system, the input data set by concatenating the dense output, one or more continuous features, and spatial information.

16. A computer system comprising:

a processor; and

a memory having programming instructions stored thereon, which, when executed by the processor, causes a computing system to perform operations comprising:

receiving match data for a sporting match;

extracting a plurality of parameters associated with an event of a play-by-play event sequence from the match data;

generating an input data set from the plurality of extracted parameters;

generating a multi-modal prediction based on the input data set; and

generating one or more graphical representations of the multi-modal prediction for the event of the play-by-play event sequence.

17. The computer system of claim 16 , the operations further comprising:

receiving the match data from one or more tracking systems.

18. The computer system of claim 16 , wherein generating the input data set includes:

transforming the match data into one or more segmented data sets;

selecting a subset of one or more segmented data sets; and

creating a dense representation of the subset of one or more segmented data sets.

19. The computer system of claim 18 , wherein generating the input data set further comprises:

providing each of the one or more segmented data sets to one or more embedding layers;

receiving dense output from the one or more embedding layers; and

generating the input data set by concatenating the dense output, one or more continuous features, and spatial information.

20. The computer system of claim 19 , wherein the one or more continuous features include a score difference, a remaining time, or a playing surface position.

Assignments (2)
SECURITY INTEREST Recorded Apr 14, 2026
From: STATS LLC
To: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
Reel/Frame 075390/0491 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 8, 2024
From: HOLBROOK, MATTHEW; HOBBS, JENNIFER; LUCEY, PATRICK JOSEPH
To: STATS LLC
Reel/Frame 066046/0379 →
Continuity (4)
Continuation 18175262 · Feb 27, 2023
Continuation 16804914 · Feb 28, 2020
Provisional Application 62812511 · Mar 1, 2019
Related Publication 20240160921A1 · May 16, 2024
References Cited (31)
US 9721086B2 · Shear · 2017 [cited by examiner]
US 20080235029A1 · Cross · 2008 [cited by examiner]
US 20080281444A1 · Krieger et al. · 2008 [cited by applicant]
US 20140201126A1 · Zadeh · 2014 [cited by examiner]
US 20150213361A1 · Gamon · 2015 [cited by examiner]
US 20170165570A1 · Lucey et al. · 2017 [cited by applicant]
US 20180001215A1 · Lazarus et al. · 2018 [cited by applicant]
US 20180204111A1 · Zadeh et al. · 2018 [cited by applicant]
US 20190087056A1 · Shults · 2019 [cited by examiner]
US 20190228290A1 · Ruiz et al. · 2019 [cited by applicant]
US 20190251743A1 · Koyama et al. · 2019 [cited by applicant]
US 20190384790A1 · Bequet et al. · 2019 [cited by applicant]
CN 111954564A · 2020 [cited by examiner]
Christopher Bishop, Mixture Density Networks, 1994, Neural Computing Research Group Report: NCRG/94/0041 Available from: http://www.ncrg.aston.ac.uk/ pp. 1-25. (Year: 1994). [cited by examiner]
Beuoy, “Updated NBA Win Probability Calculator”, inpredictable, https://www.inpredictable.com/2015/02/updated-nba-win-probability-calculator.html, Feb. 6, 2015. [cited by applicant]
Bishop, “Mixture density networks”, Neural Computing Research Group Report:004, Feb. 1994. [cited by applicant]
Caruana, “Multitask Learning”, Machine Learning, 28, Kluwer Academic Publishers, pp. 41-75, 1997. [cited by applicant]
Cervone, et al., “POINTWISE: Predicting points and valuing decisions in real time with NBA optical tracking data”, Proceedings of the 8th MIT Sloan Sports Analytics Conference, vol. 28, Feb. 2014. [cited by applicant]
Cheng, et al., “Wide & deep learning for recommender systems”, Proceedings of the 1st workshop on deep learning for recommender systems, Sep. 2016. [cited by applicant]
Ellefsen, et al., “How do Mixture Density RNNs Predict the Future?” arXiv preprint arXiv:1901.07859, Jan. 23, 2019, 10 pages. [cited by applicant]
Football Outsiders, “Methods to Our Madness”, EdjSports, LLC, https://www.footballoutsiders.com/info/methods, printed Apr. 17, 2020. [cited by applicant]
Ganguly, et al., “The problem with win probability”, 12th MIT Sloan Sports Analytics Conference, 2018. [cited by applicant]
Goldsberry, “Courtvision: New visual and spatial analytics for the NBA”, 2012 MIT Sloan Sports Analytics Conference, vol. 9, 2012. [cited by applicant]
Harmon, et al., “Predicting Shot Making in Basketball Learnt from Adversarial Multiagent Trajectories”, arXiv preprint arXiv: 1609.04849, Dec. 28, 2017, 23 pages. [cited by applicant]
Lucey, et al. “Quality vs Quantity: Improved shot prediction in soccer using strategic features from spatiotemporal data”, 8th Snnual MIT Sloan Sports Analytics Conference, Feb. 2015. [cited by applicant]
Miller, et al., “Factorized point process intensities: A spatial analysis of professional basketball”, International conference on machine learning, 2014. [cited by applicant]
Pelechrinis, “iWinrNFL: A Simple and Well-Calibrated In-Game NFL Win Probability Model”, Arxiv: Statistical Applications, Apr. 2017. [cited by applicant]
Sha, et al., “Chalkboarding: A New Spatiotemporal Query Paradigm for Sports Play Retrieval”, IUI 2016: Information Retrieval and Search, Mar. 2016, pp. 336-347. [cited by applicant]
Slowinski, “What is War?”, Fangraphs, http://www.fangraphs.com/library/misc/war/, Feb. 15, 2010. [cited by applicant]
Yue, et al., “Learning fine-grained spatial models for dynamic sports play prediction”, IEEE international conference on data mining, 2014. [cited by applicant]
Zhao, et al., “Applying deep bidirectional LSTM and mixture density network for basketball trajectory prediction”, Science Direct, Optik, vol. 158, Apr. 2018, pp. 266-272. [cited by applicant]
Cited By (1)
US 12,515,100