IP Library Granted Patent US 11,040,729
Granted Patent B2
US 11,040,729 · App. 17/048,904 · Granted Jun 22, 2021

Probabilistic object tracking and prediction framework

Inventors: Yue Zhao (Sunnyvale, CA); Christopher Ostafew (Mountain View, CA); Ali Mortazavi (Walnut Creek, CA); Liam Pedersen (San Francisco, CA)
Assignee: Nissan North America, Inc.
B60W60/0027B60W50/00G05D1/0088G05D1/0214G08G1/16H04L67/12B60W2050/0031B60W2554/4041B60W2554/4045G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,040,729
App. No.
17/048,904
Granted
Jun 22, 2021
Kind
B2
Abstract

A method for world objects tracking and prediction by an autonomous vehicle includes receiving, from sensors of the AV, a first observation data; associating the first observation data with a first world object; determining hypotheses for the first world object; determining a respective hypothesis likelihood of each of the hypotheses indicating a likelihood that the first world object follows the intention; determining, for at least one hypothesis of the hypotheses, a respective state; and in response to a query, providing a hypothesis of the hypotheses based on the respective hypothesis likelihood of each of the hypotheses. A hypothesis corresponds to an intention of the first world object and the respective state includes predicted positions of the first world object.

Claims (75)

1. A method for world objects tracking and prediction by an autonomous vehicle (AV), comprising:

receiving, from sensors of the AV, a first observation data;

associating the first observation data with a first world object by:

determining, using a gating area around the first observation data, world objects within the gating area, the world objects being users of a vehicle transportation network and are maintained in a module of the AV; and

determining an association likelihood that the first observation data corresponds to the first world object;

determining hypotheses for the first world object, wherein a hypothesis corresponds to an intention of the first world object;

determining a respective hypothesis likelihood of each of the hypotheses indicating a likelihood that the first world object follows the intention;

determining, for at least one hypothesis of the hypotheses, a respective state, wherein the respective state comprises predicted positions of the first world object; and

in response to a query, providing a hypothesis of the hypotheses based on the respective hypothesis likelihood of each of the hypotheses.

2. The method of claim 1 , wherein a hypothesis of the hypotheses is determined using map information.

3. The method of claim 1 , further comprising:

receiving, from the sensors of the AV, a second observation data;

associating the second observation data with the first world object; and

updating the respective state based on the second observation data.

4. The method of claim 1 , further comprising:

predicting, based on a motion model of the first world object, a respective trajectory for at least some of the hypotheses.

5. The method of claim 4 , wherein predicting, based on the motion model of the first world object, the respective trajectory comprises:

determining a reference driveline for the first world object; and

predicting the respective trajectory, based on the motion model, such that a current predicted path of the first world object coincides with the reference driveline.

6. The method of claim 4 , further comprising:

receiving, from the sensors of the AV, a second observation data; and

associating the second observation data with one of the first world object and a second world object.

7. The method of claim 6 , further comprising:

updating the respective hypothesis likelihood of each of the hypotheses based on the association likelihood; and

updating the respective state of the first world object.

8. The method of claim 1 , further comprising:

receiving, from the sensors of the AV, second observation data;

on a condition that the second observation data meeting a criterion, associating the second observation data with a second world object; and

on a condition that the second observation data not meeting the criterion, ignoring the second observation data.

9. The method of claim 1 , wherein the association likelihood is determined in a case that the first world object is included in the world objects.

10. A system for world objects tracking and prediction by an autonomous vehicle (AV), comprising:

a processor configured to:

receive, from sensors of the AV, a first observation data;

associate the first observation data with a first world object by, wherein to associate the first observation data with the first world object comprises to:

determine, using a gating area around the first observation data, world objects within the gating area; and

in a case that the first world object is included in the world objects, determine an association likelihood that the first observation data corresponds to the first world object;

determine hypotheses for the first world object, wherein a hypothesis corresponds to a possible path of the first world object;

determine a respective hypothesis likelihood of each of the hypotheses indicating a likelihood that the first world object follows the possible path;

determine, for at least one hypothesis of the hypotheses, a respective state, wherein the respective state comprises predicted positions of the first world object; and

in response to a query, provide a hypothesis of the hypotheses based on the respective hypothesis likelihood of each of the hypotheses.

11. The system of claim 10 , wherein a hypothesis of the hypotheses is determined using map information.

12. The system of claim 10 , wherein the processor is further configured to:

receive, from the sensors of the AV, a second observation data;

associate the second observation data with the first world object; and

update the respective state based on the second observation data.

13. The system of claim 10 , wherein the processor is further configured to:

predict, based on a motion model of the first world object, a respective trajectory for at least some of the hypotheses.

14. The system of claim 13 , wherein to predict, based on the motion model of the first world object, the respective trajectory comprises to:

determine a reference driveline for the first world object; and

predict the respective trajectory, based on the motion model, such that a current predicted path of the first world object coincides with the reference driveline.

15. The system of claim 13 , wherein the processor is further configured to:

receive, from the sensors of the AV, a second observation data; and

associate the second observation data with one of the first world object and a second world object.

16. The system of claim 15 , wherein the processor is further configured to:

update the respective hypothesis likelihood of each of the hypotheses based on the association likelihood; and

update the respective state of the first world object.

17. The system of claim 10 , wherein the processor is further configured to:

receive, from the sensors of the AV, second observation data;

on a condition that the second observation data meeting a criterion, associate the second observation data with a second world object; and

on a condition that the second observation data not meeting the criterion, ignore the second observation data.

18. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of instructions for world objects tracking and prediction by an autonomous vehicle (AV), the instructions comprising instructions comprising to:

create and maintain respective hypotheses for real-world objects, wherein the world objects being, along with the AV, users of a vehicle transportation network;

predict respective trajectories for the real-world object;

associate a received sensor observation with at least one of the real-world objects by instructions to:

determine, using a gating area around the received sensor observation, world objects within the gating area; and

in a case that the at least one of the real-world objects is included in the world objects, determine an association likelihood that the received sensor observation corresponds to the at least one of the real-world objects;

update a respective likelihood of at least some of the respective hypotheses; and

update a respective state associated with at least some of the respective hypotheses.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions further comprise instructions to:

receive a new sensor observation; and

create a new real-world object based on the new sensor observation.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions further comprise instructions to:

receive, from the sensors of the AV, second observation data;

when the second observation data meet a criterion, associate the second observation data with a second world object; and

when the second observation data do not meet the criterion, ignore the second observation data.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2021
From: NISSAN NORTH AMERICA, INC.
To: NISSAN MOTOR CO., LTD.
Reel/Frame 057337/0758 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 25, 2021
From: RENAULT S.A.S.
To: NISSAN NORTH AMERICA, INC.
Reel/Frame 055713/0449 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 19, 2020
From: NISSAN NORTH AMERICA, INC.
To: NISSAN NORTH AMERICA, INC.; RENAULT S.A.S.
Reel/Frame 054098/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 19, 2020
From: ZHAO, YUE; OSTAFEW, CHRISTOPHER; MORTAZAVI, ALI; PEDERSEN, LIAM
To: NISSAN NORTH AMERICA, INC.
Reel/Frame 054401/0911 →
Continuity (1)
Related Publication 20210086795A1 · Mar 25, 2021