IP Library Granted Patent US 10,817,752
Granted Patent B2
US 10,817,752 · App. 15/994,873 · Granted Oct 27, 2020

Virtually boosted training

Inventors: Wadim Kehl (San Francisco, CA); German Ros Sanchez (Mountain View, CA)
Assignee: TOYOTA RESEARCH INSTITUTE, INC.
G06K9/6259G06K9/00805G06K9/6262G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,817,752
App. No.
15/994,873
Granted
Oct 27, 2020
Kind
B2
Abstract

A method for training a machine learning model includes receiving real data comprising a real element in a real environment. The training also includes annotating the real element with a first annotation based on predicted attributes of the real element. The first annotation having a first format. The training further includes converting the first format of the first annotation to a second format corresponding to a ground truth annotation of the real element. The training still further includes adjusting parameters of the machine learning model to minimize a difference between values of the ground truth annotation of the real element and the converted first annotation.

Claims (40)

1. A method for training a machine learning model, comprising:

receiving real data comprising a real element in a real environment;

annotating the real element with a first annotation based on predicted attributes of the real element, the first annotation having a first format;

determining a second format based on an existing format of a ground truth annotation of the real element, the first format and the second format representing different formats of a same annotation type;

converting the first format of the first annotation to the determined second format; and

adjusting parameters of the machine learning model to minimize a difference between values of the ground truth annotation of the real element and the converted first annotation.

2. The method of claim 1 , further comprising:

receiving simulated data comprising a simulated element in a simulated environment; and

annotating the real element with a second annotation based on predicted attributes of the simulated element, the second annotation having the first format.

3. The method of claim 2 , further comprising adjusting the parameters to minimize a difference between values of a ground truth annotation of the simulated element and the second annotation.

4. The method of claim 1 , in which the first format is a three-dimensional bounding box and the second format is a two-dimensional bounding box.

5. The method of claim 1 , in which the ground truth annotation of the real element is annotated by a human annotator.

6. The method of claim 1 , further comprising planning a route for a robotic device via the adjusted machine learning model.

7. An apparatus for training a machine learning model, comprising:

a memory, and

at least one processor coupled to the memory, the at least one processor configured:

to receive real data comprising a real element in a real environment;

to annotate the real element with a first annotation based on predicted attributes of the real element, the first annotation having a first format;

to determine a second format based on an existing format of a ground truth annotation of the real element, the first format and the second format representing different formats of a same annotation type;

to convert the first format of the first annotation to the determined second format; and

to adjust parameters of the machine learning model to minimize a difference between values of the ground truth annotation of the real element and the converted first annotation.

8. The apparatus of claim 7 , in which the at least one processor is further configured:

to receive simulated data comprising a simulated element in a simulated environment; and

to annotate the real element with a second annotation based on predicted attributes of the simulated element, the second annotation having the first format.

9. The apparatus of claim 8 , in which the at least one processor is further configured to adjust the parameters to minimize a difference between values of a ground truth annotation of the simulated element and the second annotation.

10. The apparatus of claim 7 , in which the first format is a three-dimensional bounding box and the second format is a two-dimensional bounding box.

11. The apparatus of claim 7 , in which the ground truth annotation of the real element is annotated by a human annotator.

12. The apparatus of claim 7 , in which the at least one processor is further configured to plan a route for a robotic device via the adjusted machine learning model.

13. A non-transitory computer-readable medium having program code recorded thereon for training a machine learning model, the program code executed by a processor and comprising:

program code to receive real data comprising a real element in a real environment;

program code to annotate the real element with a first annotation based on predicted attributes of the real element, the first annotation having a first format;

program code to determine a second format based on an existing format of a ground truth annotation of the real element, the first format and the second format representing different formats of a same annotation type;

program code to convert the first format of the first annotation to the determined second format; and

program code to adjust parameters of the machine learning model to minimize a difference between values of the ground truth annotation of the real element and the converted first annotation.

14. The non-transitory computer-readable medium of claim 13 , in which the program code further comprises:

program code to receive simulated data comprising a simulated element in a simulated environment; and

program code to annotate the real element with a second annotation based on predicted attributes of the simulated element, the second annotation having the first format.

15. The non-transitory computer-readable medium of claim 14 , in which the program code further comprises program code to adjust the parameters to minimize a difference between values of a ground truth annotation of the simulated element and the second annotation.

16. The non-transitory computer-readable medium of claim 13 , in which the first format is a three-dimensional bounding box and the second format is a two-dimensional bounding box.

17. The non-transitory computer-readable medium of claim 13 , in which the ground truth annotation of the real element is annotated by a human annotator.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 23, 2020
From: TOYOTA RESEARCH INSTITUTE, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 054450/0413 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 5, 2018
From: KEHL, WADIM; ROS SANCHEZ, GERMAN
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 045994/0112 →
Continuity (1)
Related Publication 20190370606A1 · Dec 5, 2019