IP Library Granted Patent US 10,997,461
Granted Patent B2
US 10,997,461 · App. 16/265,729 · Granted May 4, 2021

Generating ground truth for machine learning from time series elements

Inventors: Ashok Kumar Elluswamy (Sunnyvale, CA); Matthew Bauch (San Francisco, CA); Christopher Payne (San Francisco, CA); Andrej Karpathy (San Francisco, CA); Joseph Polin (San Francisco, CA)
Assignee: Tesla, Inc.
G06K9/6255G05D1/0221G06K9/00798G06N3/08G06N3/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,997,461
App. No.
16/265,729
Filed
Feb 1, 2019
Granted
May 4, 2021
Kind
B2
Examiner
FLORES, LEON
Art Unit
2661
USPC
382/159
Abstract

Sensor data, including a group of time series elements, is received. A training data set is determined, including by determining for at least a selected time series element in the group of time series elements a corresponding ground truth. The corresponding ground truth is based on a plurality of time series elements in the group of time series elements. A processor is used to train a machine learning model using the training dataset.

Claims (38)

1. A method, comprising:

receiving sensor data including a group of time series elements captured at respective times within a period of time;

determining a ground truth for the group of time series elements, wherein determining the ground truth comprises:

identifying, for individual time series elements, respective portions of the individual time series elements to form the ground truth, and

generating the ground truth based on the identified portions; and

using a processor to train a machine learning model using a training dataset comprising the determined ground truth and a selected time series element in the group of time series elements, wherein the machine learning model is trained to output the ground truth for the group of time series elements based on an input of the selected time series element.

2. The method of claim 1 , wherein the determined ground truth is associated with a vehicle lane line.

3. The method of claim 2 , wherein identifying respective portions of the individual time series elements comprises identifying different portions of the vehicle lane line from different elements of the group of time series elements.

4. The method of claim 1 , wherein the group of time series elements are used to identify locations of a vehicle lane line in the selected time series element.

5. The method of claim 1 , wherein each element of the group of time series elements includes an image associated with a corresponding timestamp.

6. The method of claim 1 , wherein the ground truth is determined using odometry data associated with the group of time series elements.

7. The method of claim 6 , wherein the odometry data includes a vehicle position data and a vehicle orientation data.

8. The method of claim 6 , wherein the odometry data identifies a first change in a vehicle position and a second change in a vehicle orientation.

9. The method of claim 1 , wherein the determined ground truth represents a three-dimensional trajectory of a lane line.

10. The method of claim 9 , wherein the three-dimensional trajectory is represented as a parametric curve.

11. The method of claim 1 , wherein the determined ground truth is associated with a predicted path of a second vehicle different from a first vehicle that includes sensors that captured the sensor data.

12. The method of claim 11 , wherein the second vehicle is identified as entering a lane of the first vehicle.

13. The method of claim 1 , wherein the determined ground truth is associated with a distance of an object.

14. The method of claim 13 , wherein the object is an obstacle, a moving vehicle, a stationary vehicle, or a barrier.

15. The method of claim 13 , wherein the distance of the object is determined based on radar data associated with the group of time series elements.

16. The method of claim 1 , wherein a number of the elements included in the group of time series elements is based on a distance traveled.

17. The method of claim 1 , wherein a number of the elements included in the group of time series elements is based on an average vehicle speed.

18. The method of claim 1 , wherein identifying a portion of an individual time series element is based on an accuracy measure associated with a feature depicted in the ground truth.

19. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions which when executed by a processor, cause the processor to:

receive sensor data including a group of time series elements;

determine a ground truth for the group of time series elements, wherein to determine the ground truth the computer instructions cause the processor to:

identify, for individual time series elements, respective portions of the individual time series elements to form the ground truth, and

generate the ground truth based on the identified portions; and

form a training dataset, wherein the training data set is limited to the determined ground truth for the group of time series elements and a selected time series element, wherein the selected time series element is a single time series element of the group of time series elements,

and wherein the training dataset is configured to train a machine learning model to output the ground truth for the group of time series elements based on an input of the single time series element.

20. A system, comprising:

a processor; and

a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:

receive sensor data including a group of time series elements;

determine a ground truth for the group of time series elements, wherein to determine the ground truth the instructions cause the processor to:

identify, for individual time series elements, respective portions of the individual time series elements to form the ground truth, and

generate the ground truth based on the identified portions; and

train a machine learning model using a training dataset comprising the determined ground truth and a selected time series element in the group of time series elements, wherein the machine learning model is trained to output the ground truth for the group of time series elements based on an input of the selected time series element.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 1, 2019
From: ELLUSWAMY, ASHOK KUMAR; BAUCH, MATTHEW; PAYNE, CHRISTOPHER; KARPATHY, ANDREJ; POLIN, JOSEPH
To: TESLA, INC.
Reel/Frame 049052/0861 →
Continuity (1)
Related Publication 20200250473A1 · Aug 6, 2020
Cited By (4)
US 12,223,428 US 12,572,522 US 12,607,738 US 12,693,833