IP Library Granted Patent US 11,049,265
Granted Patent B2
US 11,049,265 · App. 16/406,242 · Granted Jun 29, 2021

Balancing diversity and precision of generative models with complementary density estimators

Inventors: Paul Vernaza (Sunnyvale, CA); Nicholas Rhinehart (Pittsburgh, PA); Anqi Liu (Pasadena, CA); Kihyuk Sohn (Fremont, CA)
G06T7/251G06K9/00201G06K9/00805G06N3/08G06N7/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,049,265
App. No.
16/406,242
Granted
Jun 29, 2021
Kind
B2
Abstract

Systems and methods for training and evaluating a deep generative model with an architecture consisting of two complementary density estimators are provided. The method includes receiving a probabilistic model of vehicle motion, and training, by a processing device, a first density estimator and a second density estimator jointly based on the probabilistic model of vehicle motion. The first density estimator determines a distribution of outcomes and the second density estimator estimates sample quality. The method also includes identifying by the second density estimator spurious modes in the probabilistic model of vehicle motion. The probabilistic model of vehicle motion is adjusted to eliminate the spurious modes.

Claims (44)

1. A method for training and evaluating a deep generative model with an architecture consisting of two complementary density estimators, comprising:

receiving a probabilistic model of vehicle motion;

training, by a processing device, a first density estimator and a second density estimator jointly based on the probabilistic model of vehicle motion, wherein the first density estimator determines a distribution of outcomes and the second density estimator estimates sample quality;

identifying by the second density estimator spurious modes in the probabilistic model of vehicle motion; and

adjusting the probabilistic model of vehicle motion to eliminate the spurious modes.

2. The method as recited in claim 1 , wherein adjusting the probabilistic model of vehicle motion further comprises:

generating an adjusted model that assigns high probability to plausible paths in a scene while ensuring that every prediction generated by the adjusted model is also likely under a true distribution of future paths.

3. The method as recited in claim 1 , wherein the probabilistic model of vehicle motion includes continuous, high-dimensional, structured data based on paths of intelligent agents.

4. The method as recited in claim 1 , further comprising:

outputting a message to display a warning on an associated display based on the probabilistic model of vehicle motion.

5. The method as recited in claim 1 , further comprising:

training the probabilistic model of vehicle motion to balance dual objectives of generating diverse samples while ensuring high quality samples.

6. The method as recited in claim 1 , further comprising:

determining potential modes of future behavior; and

determining the distribution of outcomes based on the potential modes of future behavior.

7. The method as recited in claim 1 , wherein training the first density estimator and the second density estimator further comprises:

training a network to obtain improved training efficiency, stability, and a balance of diversity and precision in generated examples, while producing an interpretable model.

8. The method as recited in claim 1 , wherein the second estimator is trained by optimizing a variational lower bound based on Fenchel duality.

9. The method as recited in claim 1 , wherein an optimal value of a model associated with the second estimator is independent of the probabilistic model of vehicle motion.

10. A computer system for training and evaluating a deep generative model with an architecture consisting of two complementary density estimators, comprising:

a processor device operatively coupled to a memory device, the processor device being configured to:

receive a probabilistic model of vehicle motion;

train a first density estimator and a second density estimator jointly based on the probabilistic model of vehicle motion, wherein the first density estimator determines a distribution of outcomes and the second density estimator estimates sample quality;

identify by the second density estimator spurious modes in the probabilistic model of vehicle motion; and

adjust the probabilistic model of vehicle motion to eliminate the spurious modes.

11. The system as recited in claim 10 , wherein, when adjusting the probabilistic model of vehicle motion, the processor device is further configured to:

generate an adjusted model that assigns high probability to plausible paths in a scene while ensuring that every prediction generated by the adjusted model is also likely under a true distribution of future paths.

12. The system as recited in claim 10 , wherein the probabilistic model of vehicle motion includes continuous, high-dimensional, structured data based on paths of intelligent agents.

13. The system as recited in claim 10 , wherein the processor device is further configured to:

output a message to display a warning on an associated display based on the probabilistic model of vehicle motion.

14. The system as recited in claim 10 , wherein the processor device is further configured to:

train the probabilistic model of vehicle motion to balance dual objectives of generating diverse samples while ensuring high quality samples.

15. The system as recited in claim 10 , wherein the processor device is further configured to:

determine potential modes of future behavior; and

determine the distribution of outcomes based on the potential modes of future behavior.

16. The system as recited in claim 10 , wherein, when training the first density estimator and the second density estimator, the processor device is further configured to:

train a network to obtain improved training efficiency, stability, and a balance of diversity and precision in generated examples, while producing an interpretable model.

17. The system as recited in claim 10 , wherein the second estimator is trained by optimizing a variational lower bound based on Fenchel duality.

18. The system as recited in claim 10 , wherein an optimal value of a model associated with the second estimator is independent of the probabilistic model of vehicle motion.

19. A computer program product for training and evaluating a deep generative model with an architecture consisting of two complementary density estimators, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to perform the method comprising:

receiving a probabilistic model of vehicle motion;

training, by a processing device, a first density estimator and a second density estimator jointly based on the probabilistic model of vehicle motion, wherein the first density estimator determines a distribution of outcomes and the second density estimator estimates sample quality;

identifying by the second density estimator spurious modes in the probabilistic model of vehicle motion; and

adjusting the probabilistic model of vehicle motion to eliminate the spurious modes.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2021
From: NEC LABORATORIES AMERICA, INC.
To: NEC CORPORATION
Reel/Frame 056231/0358 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 8, 2019
From: VERNAZA, PAUL; RHINEHART, NICHOLAS; LIU, ANQI; SOHN, KIHYUK
To: NEC LABORATORIES AMERICA, INC.
Reel/Frame 049112/0722 →
Continuity (3)
Provisional Application 62672571 · May 16, 2018
Provisional Application 62796658 · Jan 25, 2019
Related Publication 20190355134A1 · Nov 21, 2019