IP Library Granted Patent US 11,361,246
Granted Patent B2
US 11,361,246 · App. 16/133,050 · Granted Jun 14, 2022

Methods for automating aspects of machine learning, and related systems and apparatus

Inventor: Michael Schmidt (Boston, MA)
Assignee: DataRobot, Inc.
G06N20/00G06F16/2428G06F16/2453G06F40/18G06N7/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,361,246
App. No.
16/133,050
Granted
Jun 14, 2022
Kind
B2
Abstract

Various systems and methods provide an intuitive user interface that enables automatic specification of queries and constraints for analysis by ML component. Various implementations provide methodologies for automatically formulating machine learning (“ML”) and optimization queries. The automatic generation of ML and/or optimization queries can be configured to use examples to facilitate formulation of ML and optimization queries. One example method includes accepting input data specifying variables and data values associated with the variables. Within the input data any unspecified data records are identified, and a relationship between the variables specified in the input data and a variable associated with the at least one unspecified data record is automatically determined. The relationship can be automatically determined based on training data contained within the input data. Once a relationship is established a ML problem can be automatically generated.

Claims (42)

1. A method, comprising:

accessing, by a computer system, time series data comprising a sequence of data records, each of the data records in the sequence corresponding to a respective time and indicating data values of a first variable and a second variable for the respective time;

generating, by the computer system via application of machine learning to the sequence of data records, a function that computes a data value of the first variable for a first time based at least in part on respective data values of at least the second variable for a plurality of times prior to the first time; and

determining, by the computer system using the function generated via application of machine learning to the sequence of data records, a data value of the first variable for a second time subsequent to one or more times corresponding to any of the data records in the sequence.

2. A system, comprising:

one or more processors connected to memory to:

access time series data comprising a sequence of data records, each of the data records in the sequence corresponding to a respective time and indicating data values of a first variable and a second variable for the respective time;

generate, via application of machine learning to the sequence of data records, a function that computes a data value of the first variable for a first time based at least in part on respective data values of at least the second variable for a plurality of times prior to the first time; and

determine, using the function generated via application of machine learning to the sequence of data records, a data value of the first variable for a second time subsequent to one or more times corresponding to any of the data records in the sequence.

3. The method of claim 1 , comprising:

determining, by the computer system, a confidence value associated with the data value of the first variable for the second time subsequent to the one or more times determined using the function.

4. The method of claim 1 , comprising:

converting, by the computer system, one or more values of the second variable in the sequence of data records into binary values.

5. The method of claim 1 , wherein the machine learning comprises symbolic regression, comprising:

training, by the computer system, a model using the machine learning; and

generating, by the computer system, the function using the model.

6. The method of claim 1 , comprising:

training, by the computer system using the machine learning, a plurality of candidate models for the function that computes the data value of the first variable for the first time and the respective data values of at least the second variable for the plurality of times prior to the first time.

7. The method of claim 1 , comprising:

determining, by the computer system using the function and based at least on part on the data value of the first variable determined for the second time, a data value of the first variable for a third time subsequent to the second time.

8. The method of claim 1 , comprising:

providing, by the computer system, the function generated via application of the machine learning for display via a graphical user interface.

9. The system of claim 2 , wherein the one or more processors are further configured to determine a confidence value associated with the data value of the first variable for the second time subsequent to the one or more times determined using the function.

10. The system of claim 2 , wherein the one or more processors are further configured to convert one or more values of the second variable in the sequence of data records into binary values.

11. The system of claim 2 , wherein the machine learning comprises symbolic regression, and the one or more processors are further configured to:

train a model using the machine learning; and

generate the function using the model.

12. The system of claim 2 , wherein the one or more processors are further configured to train, using the machine learning, a plurality of candidate models for the function that computes the data value of the first variable for the first time and the respective data values of at least the second variable for the plurality of times prior to the first time.

13. The system of claim 2 , wherein the one or more processors are further configured to determine, using the function and based at least on part on the data value of the first variable determined for the second time, a data value of the first variable for a third time subsequent to the second time.

14. The system of claim 2 , wherein the one or more processors are further configured to provide the function generated via application of the machine learning for display via a graphical user interface.

15. The method of claim 3 , comprising:

providing, by the computer system, an indication of the confidence value for display via a display device.

16. The method of claim 3 , comprising:

providing, by the computer system, a visual indication of the confidence value that indicates a higher level of confidence relative to a lower level of confidence associated with a second value.

17. The method of claim 6 , comprising:

determining, by the computer system, respective confidence levels associated with the plurality of candidate models; and

selecting, by the computer system, a candidate model from the plurality of candidate models based on the respective confidence levels to use to generate the function.

18. The system of claim 9 , wherein the one or more processors are further configured to provide an indication of the confidence value for display via a display device.

19. The system of claim 9 , wherein the one or more processors are further configured to provide a visual indication of the confidence value that indicates a higher level of confidence relative to a lower level of confidence associated with a second value.

20. The system of claim 12 , wherein the one or more processors are further configured to:

determine respective confidence levels associated with the plurality of candidate models; and

select a candidate model from the plurality of candidate models based on the respective confidence levels to use to generate the function.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Apr 7, 2025
From: CITIBANK, N.A.
To: DATAROBOT, INC.; ALGORITHMIA, INC.; DULLES RESEARCH, LLC
Reel/Frame 070750/0866 →
SECURITY INTEREST Recorded Mar 22, 2023
From: DATAROBOT, INC.; ALGORITHMIA, INC.; DULLES RESEARCH, LLC
To: CITIBANK, N.A.
Reel/Frame 063263/0926 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2022
From: SCHMIDT, MICHAEL
To: NUTONIAN, INC.
Reel/Frame 058914/0787 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2022
From: NUTONIAN, INC.
To: DATAROBOT, INC.
Reel/Frame 058914/0798 →
Continuity (10)
Continuation 15351002 · Nov 14, 2016
Continuation 14202780 · Mar 10, 2014
Continuation In Part 14016287 · Sep 3, 2013
Continuation In Part 14016300 · Sep 3, 2013
Provisional Application 61695637 · Aug 31, 2012
Provisional Application 61695660 · Aug 31, 2012
Provisional Application 61695637 · Aug 31, 2012
Provisional Application 61695660 · Aug 31, 2012
Provisional Application 61778451 · Mar 13, 2013
Related Publication 20190220772A1 · Jul 18, 2019