IP Library › Granted Patent US 12,236,327
Granted Patent B1
US 12,236,327 · App. 18/536,876 · Granted Feb 25, 2025

Automated system and method for modeling the behavior of vehicles and other agents

Inventors: Jiajun Zhu (Palo Alto, CA); David I. Ferguson (San Francisco, CA); Dmitri A. Dolgov (Los Altos, CA)
Assignee: Waymo LLC
G06N20/00B60W30/08B60W30/09B60W30/095B60W30/0956B60W50/0097B60W60/00133B60W60/00274B60W60/00276G05D1/0088G05D1/0214G06N3/00G06V20/56G06V20/58G06V20/582G06V20/584G06V20/588G08G1/09623G08G1/09626G08G1/096725G08G1/096741G08G1/096775G08G1/164G08G1/165G08G1/166B60W2420/403B60W2420/408B60W2554/00B60W2554/20B60W2554/4023B60W2554/4029B60W2554/80B60W2555/20B60W2556/10G01R19/25G05D1/0212G05D1/0289G06V10/255
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,236,327
App. No.
18/536,876
Granted
Feb 25, 2025
Kind
B1
Abstract

A method and apparatus are provided for determining one or more behavior models used by an autonomous vehicle to predict the behavior of detected objects. The autonomous vehicle may collect and record object behavior using one or more sensors. The autonomous vehicle may then communicate the recorded object behavior to a server operative to determine the behavior models. The server may determine the behavior models according to a given object classification, actions of interest performed by the object, and the object's perceived surroundings.

Claims (35)

1. A method of maneuvering a vehicle, the method comprising:

receiving, by one or more processors, object data identifying an object in an environment of the vehicle;

identifying, by the one or more processors, a behavior of interest corresponding to an action taken by the object based on the object data;

determining, by the one or more processors, a plurality of probabilities for potential actions of the object using a behavior model, the behavior model configured to predict how another object having a same classification as the object will behave;

updating, by the one or more processors, the behavior model by adjusting the plurality of probabilities based on the behavior of interest; and

maneuvering, by the one or more processors, the vehicle based on the updated behavior model and the behavior of interest.

2. The method of claim 1 , wherein the behavior model includes paths of travel for one or more previously monitored objects.

3. The method of claim 1 , wherein maneuvering the vehicle includes avoiding a trajectory for at least one of the potential actions.

4. The method of claim 1 , wherein identifying the behavior of interest is further based on map information identifying a road element corresponding to a location of the object.

5. The method of claim 4 , wherein the road element comprises a first lane of traffic, and wherein the action taken indicates that the object has changed from a second lane of traffic to the first lane of traffic.

6. The method of claim 1 , wherein a classification of the object corresponds to a road vehicle.

7. The method of claim 1 , wherein a classification of the object corresponds to a pedestrian.

8. The method of claim 1 , further comprising creating, by the one or more processors, an initial behavior model for the object based on the object data, when a behavior model does not exist for a classification for the object.

9. The method of claim 1 , wherein the behavior model provides information based on weather conditions.

10. A system for maneuvering a vehicle, the system comprising one or more processors configured to:

receive object data identifying an object in an environment of the vehicle;

identify a behavior of interest corresponding to an action taken by the object based on the object data;

determine a plurality of probabilities for potential actions of the object using a behavior model, the behavior model configured to predict how another object having a same classification as the object will behave;

update the behavior model by adjusting the plurality of probabilities based on the behavior of interest; and

maneuver the vehicle based on the updated behavior model and the behavior of interest.

11. The system of claim 10 , wherein the behavior model includes paths of travel for one or more previously monitored objects.

12. The system of claim 10 , wherein maneuvering the vehicle includes avoiding a trajectory for at least one of the potential actions.

13. The system of claim 10 , wherein the one or more processors are configured to identify the behavior of interest further based on map information identifying a road element corresponding to a location of the object.

14. The system of claim 13 , wherein the road element comprises a first lane of traffic, and wherein the action taken indicates that the object has changed from a second lane of traffic to the first lane of traffic.

15. The system of claim 10 , wherein a classification of the object corresponds to a road vehicle.

16. The system of claim 10 , wherein a classification of the object corresponds to a pedestrian.

17. The system of claim 10 , wherein the one or more processors are further configured to create an initial behavior model for the object based on the object data, when a behavior model does not exist for a classification for the object.

18. The system of claim 10 , wherein the behavior model provides information based on weather conditions.

19. A non-transitory, computer readable recording medium on which instructions are stored, the instructions, when executed by one or more processors, cause the one or more processors to perform a method of maneuvering a vehicle, the method comprising:

receiving object data identifying an object in an environment of the vehicle;

identifying a behavior of interest corresponding to an action taken by the object based on the object data;

determining a plurality of probabilities for potential actions of the object using a behavior model, the behavior model configured to predict how another object having a same classification as the object will behave;

updating the behavior model by adjusting the plurality of probabilities based on the behavior of interest; and

maneuvering the vehicle based on the updated behavior model and the behavior of interest.

20. The non-transitory, computer readable recording medium of claim 19 , wherein the method further comprises creating an initial behavior model for the object based on the object data, when a behavior model does not exist for a classification for the object.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2023
From: GOOGLE INC.
To: WAYMO HOLDING INC.
Reel/Frame 065845/0421 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2023
From: ZHU, JIAJUN; FERGUSON, DAVID I; DOLGOV, DMITRI A
To: GOOGLE INC.
Reel/Frame 065845/0268 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2023
From: WAYMO HOLDING INC.
To: WAYMO LLC
Reel/Frame 065845/0561 →
Continuity (4)
Continuation 17523350 · Nov 10, 2021
Continuation 16039827 · Jul 19, 2018
Continuation 15278338 · Sep 28, 2016
Continuation 13446494 · Apr 13, 2012
References Cited (39)
US 7102496B1 · Ernst, Jr. et al. · 2006 [cited by applicant]
US 8660734B2 · Zhu et al. · 2014 [cited by applicant]
US 9495874B1 · Zhu · 2016 [cited by examiner]
US 10059334B1 · Zhu · 2018 [cited by examiner]
US 10678259B1 · Ferguson · 2020 [cited by examiner]
US 11198430B1 · Zhu · 2021 [cited by examiner]
US 11878683B1 · Zhu · 2024 [cited by examiner]
US 20060041381A1 · Simon et al. · 2006 [cited by applicant]
US 20060106530A1 · Horvitz et al. · 2006 [cited by applicant]
US 20070250240A1 · Reisner · 2007 [cited by examiner]
US 20080084283A1 · Kalik · 2008 [cited by applicant]
US 20080189040A1 · Nasu et al. · 2008 [cited by applicant]
US 20080266168A1 · Aso et al. · 2008 [cited by applicant]
US 20080303696A1 · Aso et al. · 2008 [cited by applicant]
US 20100097200A1 · Hilsebecher et al. · 2010 [cited by applicant]
US 20100106418A1 · Kindo et al. · 2010 [cited by applicant]
US 20100198477A1 · Shirai · 2010 [cited by applicant]
US 20100205132A1 · Taguchi et al. · 2010 [cited by applicant]
US 20100305858A1 · Richardson · 2010 [cited by applicant]
US 20110054793A1 · Harada et al. · 2011 [cited by applicant]
US 20110071731A1 · Eidehall et al. · 2011 [cited by applicant]
US 20110087433A1 · Yester · 2011 [cited by examiner]
US 20110313664A1 · Sakai et al. · 2011 [cited by applicant]
US 20110313740A1 · Ikeda et al. · 2011 [cited by applicant]
US 20120016581A1 · Mochizuki · 2012 [cited by examiner]
US 20120059789A1 · Sakai · 2012 [cited by examiner]
US 20120083960A1 · Zhu et al. · 2012 [cited by applicant]
US 20120179518A1 · Jaipaul · 2012 [cited by applicant]
US 20120313806A1 · Katoh · 2012 [cited by examiner]
US 20130024075A1 · Zagorski · 2013 [cited by applicant]
US 20130151058A1 · Zagorski et al. · 2013 [cited by applicant]
US 20130238181A1 · James · 2013 [cited by applicant]
US 20130246020A1 · Zeng · 2013 [cited by applicant]
US 20140032017A1 · Anderson et al. · 2014 [cited by applicant]
US 20140211487A1 · Spiro · 2014 [cited by examiner]
US 20150003087A1 · Futamura · 2015 [cited by examiner]
US 20150170498A1 · Beggs · 2015 [cited by examiner]
US 20160137208A1 · Powers · 2016 [cited by examiner]
US 20190333370A1 · Malkes · 2019 [cited by examiner]