IP Library Granted Patent US 12,393,873
Granted Patent B2
US 12,393,873 · App. 18/348,217 · Granted Aug 19, 2025

Customized predictive analytical model training

Inventors: Jordan M. Breckenridge (Menlo Park, CA); Travis H. K. Green (New York, NY); Robert Kaplow (New York, NY); Wei-Hao Lin (New York, NY); Gideon S. Mann (New York, NY)
Assignee: Google LLC
G06N20/00G06F18/217G06F18/285G06N5/02G06N7/01H04L67/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,393,873
App. No.
18/348,217
Granted
Aug 19, 2025
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on one or more computer storage devices, for training predictive models. Multiple training data records are received that each include an input data portion and an output data portion. A training data type is determined that corresponds to the training data. For example, a training data type can be determined by inputting the output data portions into one or more trained predictive classifiers. In other example, the training data type can be determined by comparison of the output data portions to data formats. Based on the determined training data type, a set of training functions are identified that are compatible with the training data of the determined training data type. The training data and the identified set of training functions are used to train multiple predictive models.

Claims (46)

1. A method comprising:

sending, from a client device and to a predictive modeling server system, training data records, wherein:

each training data record includes an input data portion and an output data portion and the training data records are used to train one of a plurality of predictive models that are each of a different predictive model type;

the training data records do not specify a predictive model type to be trained on the training data records; and

the predictive modeling server is configured to compare the output data portions of the training data records to a plurality of data formats, and based on the comparison, determining a training data type and a corresponding predictive model type to select a predictive model to train on the training data records;

receiving, at the client device and from the predictive modeling server system, access to a selected predictive model that has been trained on the training data records and that executes on the predictive modeling server;

sending, from the client device to the selected predicted model executing on the predictive modeling server, input data records and a prediction request; and

receiving, from the selected predictive model, predictive output generated by applying the received input data records to the selected predictive model.

2. The method of claim 1 , wherein the training data records are sent through an HTTP web service.

3. The method of claim 1 , wherein receiving access to the selected predictive model providing comprises receiving a URL address for accessing the selected predictive model.

4. The method of claim 1 , wherein receiving access to the selected predictive model providing comprises receiving an interface configured to interface a trained model repository with a network communicatively coupled to the client device.

5. The method of claim 4 , wherein the interface is further configured to interface a training data repository with the network communicatively coupled to the client device.

6. The method of claim 1 , wherein the selected predictive model is selected at the predictive modeling server by:

generating a score for each of the plurality of trained predictive models, where each score represents an estimation of the effectiveness of the respective trained predictive model trained on the training data records; and

selecting a first trained predictive model from among the plurality of trained predictive models based on the generated scores.

7. A system comprising:

a client computer device configured to perform operations comprising:

sending, from the client device and to a predictive modeling server system, training data records, wherein:

each training data record includes an input data portion and an output data portion and the training data records are used to train one of a plurality of predictive models that are each of a different predictive model type;

the training data records do not specify a predictive model type to be trained on the training data records; and

the predictive modeling server is configured to compare the output data portions of the training data records to a plurality of data formats, and based on the comparison, determining a training data type and a corresponding predictive model type to select a predictive model to train on the training data records;

receiving, at the client device and from the predictive modeling server system, access to a selected predictive model that has been trained on the training data records and that executes on the predictive modeling server;

sending, from the client device to the selected predicted model executing on the predictive modeling server, input data records and a prediction request; and

receiving, from the selected predictive model, predictive output generated by applying the received input data records to the selected predictive model.

8. The system of claim 7 , wherein the training data records are sent through an HTTP web service.

9. The system of claim 7 , wherein receiving access to the selected predictive model providing comprises receiving a URL address for accessing the selected predictive model.

10. The system of claim 7 , wherein receiving access to the selected predictive model providing comprises receiving an interface configured to interface a trained model repository with a network communicatively coupled to the client device.

11. The system of claim 10 , wherein the interface is further configured to interface a training data repository with the network communicatively coupled to the client device.

12. The system of claim 7 , wherein the selected predictive model is selected at the predictive modeling server by:

generating a score for each of the plurality of trained predictive models, where each score represents an estimation of the effectiveness of the respective trained predictive model trained on the training data records; and

selecting a first trained predictive model from among the plurality of trained predictive models based on the generated scores.

13. A non-transitory computer-readable medium comprising instructions that, when executed by a client device, cause the client device to perform operations comprising:

sending, from the client device and to a predictive modeling server system, training data records, wherein:

each training data record includes an input data portion and an output data portion and the training data records are used to train one of a plurality of predictive models that are each of a different predictive model type;

the training data records do not specify a predictive model type to be trained on the training data records; and

the predictive modeling server is configured to compare the output data portions of the training data records to a plurality of data formats, and based on the comparison, determining a training data type and a corresponding predictive model type to select a predictive model to train on the training data records;

receiving, at the client device and from the predictive modeling server system, access to a selected predictive model that has been trained on the training data records and that executes on the predictive modeling server;

sending, from the client device to the selected predicted model executing on the predictive modeling server, input data records and a prediction request; and

receiving, from the selected predictive model, predictive output generated by applying the received input data records to the selected predictive model.

14. The non-transitory computer-readable medium of claim 13 , wherein the training data records are sent through an HTTP web service.

15. The non-transitory computer-readable medium of claim 13 , wherein receiving access to the selected predictive model providing comprises receiving a URL address for accessing the selected predictive model.

16. The non-transitory computer-readable medium of claim 13 , wherein receiving access to the selected predictive model providing comprises receiving an interface configured to interface a trained model repository with a network communicatively coupled to the client device.

17. The non-transitory computer-readable medium of claim 16 , wherein the interface is further configured to interface a training data repository with the network communicatively coupled to the client device.

18. The non-transitory computer-readable medium of claim 13 , wherein the selected predictive model is selected at the predictive modeling server by:

generating a score for each of the plurality of trained predictive models, where each score represents an estimation of the effectiveness of the respective trained predictive model trained on the training data records; and

selecting a first trained predictive model from among the plurality of trained predictive models based on the generated scores.

Assignments (3)
CHANGE OF NAME Recorded Jul 9, 2025
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 071876/0336 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED ON REEL 064200 FRAME 0635. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 9, 2023
From: BRECKENRIDGE, JORDAN M.; GREEN, TRAVIS H. K.; KAPLOW, ROBERT; LIN, WEI-HAO; MANN, GIDEON S.
To: GOOGLE INC.
Reel/Frame 064544/0389 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2023
From: BRECKENRIDGE, JORDAN M.; GREEN, TRAVIS H. K.; KAPLOW, ROBERT; LIN, WEI-HAO; MANN, GIDEON S.
To: GOOGLE LLC
Reel/Frame 064200/0635 →
Continuity (5)
Continuation 17350991 · Jun 17, 2021
Continuation 15155343 · May 16, 2016
Continuation 14295563 · Jun 4, 2014
Continuation 13170067 · Jun 27, 2011
Related Publication 20230351265A1 · Nov 2, 2023
References Cited (43)
US 5572624A · Sejnoha · 1996 [cited by examiner]
US 5692107A · Simoudis et al. · 1997 [cited by applicant]
US 7349919B2 · Russell et al. · 2008 [cited by applicant]
US 8209271B1 · Lin et al. · 2012 [cited by applicant]
US 8209274B1 · Lin et al. · 2012 [cited by applicant]
US 8229864B1 · Lin et al. · 2012 [cited by applicant]
US 8250009B1 · Breckenridge et al. · 2012 [cited by applicant]
US 8301498B1 · Cortes et al. · 2012 [cited by applicant]
US 8473431B1 · Mann et al. · 2013 [cited by applicant]
US 8762299B1 · Breckenridge et al. · 2014 [cited by applicant]
US 9489366B2 · Scott · 2016 [cited by applicant]
US 20030176931A1 · Pednault · 2003 [cited by applicant]
US 20030212851A1 · Drescher et al. · 2003 [cited by applicant]
US 20040002930A1 · Oliver · 2004 [cited by applicant]
US 20040249779A1 · Nauck · 2004 [cited by applicant]
US 20050102303A1 · Russell et al. · 2005 [cited by applicant]
US 20050114360A1 · Russell et al. · 2005 [cited by applicant]
US 20050234753A1 · Pinto et al. · 2005 [cited by applicant]
US 20050273708A1 · Motyka et al. · 2005 [cited by applicant]
US 20070219433A1 · Stupp · 2007 [cited by applicant]
US 20080008391A1 · Geva et al. · 2008 [cited by applicant]
US 20080077544A1 · Sueka · 2008 [cited by applicant]
US 20090106178A1 · Chu · 2009 [cited by applicant]
US 20120166156A1 · Meng et al. · 2012 [cited by applicant]
US 20130268467A1 · Yoon et al. · 2013 [cited by applicant]
US 20140258185A1 · Sharon · 2014 [cited by applicant]
Apte et al., “Data-intensive analytics for predictive modeling.” IBM Journal of Research and Development 47.1, Jan. 2003, 17-23. [cited by applicant]
Benediktsson et al., “Hybrid Consensus Theoretic Classification” IEEE Transactions on geoscience and remote sensing 35.4, Jul. 1997, 833-843. [cited by applicant]
Campos et al., “Data-Centric Automated Data Mining” Proceedings of the Fourth International Conference on Machine Learning and Applications (ICLMLA'05), IEEE, Dec. 2005, 8 pages. [cited by applicant]
Chandra et al. “Sibyl: a system for large scale machine learning” Keynote I PowerPoint presentation, Jul. 28, 2010, 43 pages. [cited by applicant]
Code.google.com [online], “Google Prediction API (Labs)”, Aug. 2011, retrieved on May 18, 2010, retrieved from URL <http://code.google.com/apis/predict/>, 21 pages. [cited by applicant]
Duchi et al., “Boosting with Structural Sparsity”, Proceedings of the 26th International Conference on Machine Learning, Montreal, Canada, Jun. 2009, 297-304. [cited by applicant]
Evaluating Predictive Models, 36-350, Data Mining, Principles of Data Mining, Chapter 7, Berk chapter 2, Oct. 26-28, 2001, 22 pages. [cited by applicant]
Hall et al., “The WEKA Data Mining Software: An Update”; SIGKDD Explorations, vol. 11, Issue 1, Jun. 2009, 10-18. [cited by applicant]
Hutter et al., “Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms,”, Principles and Practice of Constraint Programming, Lecture Notes in Computer Science, 2006, 15 pages. [cited by applicant]
Postema et al., “A Decision Support Tool for Tuning Parameters in a Machine Learning Algorithm”, Nanyang Technological University, 1997, 9 pages. [cited by applicant]
R-project.org [online], “R-Project web pages” Aug. 2011, retrieved on May 18, 2010, retrieved from URL <http://www.r-project.org/>, 190 pages. [cited by applicant]
Sdn.sap.com [online], “Integrating Real-Time Predictive Analytics into SAP Applications” Dec. 2009, retrieved on Sep. 29, 2011, retrieved from URL <http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/a07f… [cited by applicant]
Statistical Analysis System, “SAS Rapid Predictive Modeler” Dec. 2010, 4 pages. [cited by applicant]
Uclassify.com [online], “Uclassify web pages” Aug. 2011, retrieved on May 18, 2010, retrieved from URL <http://www.uclassify.com/>, 16 pages. [cited by applicant]
Weiss “Integrating Real-Time Predictive Analytics into SAP Applications,” SAP community Network, 2009, 10 pages. [cited by applicant]
Wikipedia.org [online], “Active learning (machine learning)” Aug. 2011, retrieved on Aug. 9, 2011, retrieved from URL <http://en.wikipedia.org/wiki/Active learning (machine learning)>, 3 pages. [cited by applicant]
Zementis.com [online], “Zementis web pages” Aug. 2011, retrieved on May 18, 2010, retrieved from URL <http://www.zementis.com/>, 34 pages. [cited by applicant]