IP Library Granted Patent US 12,444,302
Granted Patent B2
US 12,444,302 · App. 18/921,520 · Granted Oct 14, 2025

Application monologue for self-driving vehicles

Inventors: Salil Pandit (Palo Alto, CA); Nirmal Patel (Sunnyvale, CA); Renaud-Roland Hubert (Gilroy, CA); Maria Moon (Mountain View, CA); Lauren Schwendimann (Burlingame, CA); Guilherme Villar (San Francisco, CA); Ryan Powell (San Francisco, CA); Peter Crandall (San Jose, CA); Matthew Corey Hall (Cupertino, CA)
Assignee: Waymo LLC
G08G1/123G06Q50/40H04W4/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,444,302
App. No.
18/921,520
Filed
Oct 21, 2024
Granted
Oct 14, 2025
Kind
B2
Art Unit
2685
USPC
340/994
Abstract

The technology includes communicating the current status of a self-driving vehicle to users, such as passengers within the vehicle and other users awaiting pickup. Certain information about the trip and vehicle status is communicated depending on where the passenger is sitting within the vehicle or where the person awaiting pickup is located outside the vehicle. This includes disseminating the “monologue” of a vehicle operating in an autonomous driving mode to a user via an app on the user's device (e.g., mobile phone, tablet or laptop PC, wearable, or other computing device) and/or an in-vehicle user interface. The monologue includes current status information regarding driving decisions and related operations or actions. This alerts the user as to why the vehicle is taking (or not taking) a certain action, which reduces confusion and allows the user to focus on other matters.

Claims (38)

1. A method, comprising:

obtaining, by one or more processors of a computing system, vehicle status information about a vehicle operating in an autonomous driving mode, the vehicle status information comprising information about a current trip associated with a passenger;

obtaining, by the one or more processors, information about a traffic signal or a traffic sign that the vehicle is approaching in the autonomous driving mode;

determining, by the one or more processors, whether the traffic signal or the traffic sign will delay a pickup time for the passenger at a selected location by at least a threshold amount;

and

upon determining that the traffic signal or the traffic sign will delay the pickup time by the at least the threshold amount, causing, by the one or more processors, a visual representation of the traffic signal or the traffic sign to be displayed concurrently with a map and an indication of a reason for the delay in a graphical user interface of a computing device of the passenger.

2. The method of claim 1 , wherein the information about the current trip includes at least one of a time until the vehicle will arrive at the selected location, a distance between the vehicle and the selected location, information to identify the vehicle, or information about a stop along a route.

3. The method of claim 1 , wherein the passenger is awaiting pickup by the vehicle.

4. The method of claim 1 , wherein the indication of the reason for the delay includes textual information that explains the information about the traffic signal or the traffic sign.

5. The method of claim 4 , wherein causing at least one of the visual representation or the textual information to be displayed in the graphical user interface stops after the vehicle reaches or passes the traffic signal or the traffic sign.

6. The method of claim 4 , wherein the textual information identifies a color of the traffic signal or a type of the traffic sign.

7. The method of claim 1 , wherein the visual representation includes a color of the traffic signal or a visualization of the traffic sign.

8. The method of claim 1 , wherein causing the visual representation to be displayed in the graphical user interface includes determining when to display the visual representation according to a distance between the vehicle and the traffic signal or the traffic sign.

9. The method of claim 1 , wherein obtaining the information about the traffic signal or the traffic sign includes receiving the information from the vehicle during operation in the autonomous driving mode.

10. The method of claim 1 , wherein the information about the current trip includes a message from the vehicle communicating its current status.

11. The method of claim 1 , wherein:

the information about the traffic signal indicates that there is a red light; and

the visual representation of the traffic signal depicts the red light.

12. The method of claim 1 , further comprising:

upon determining that the traffic signal or the traffic sign will not delay the pickup time by the at least the threshold amount, causing, by the one or more processors, the visual representation to be displayed concurrently with the map in the graphical user interface without the indication of the reason for the delay.

13. A computing system comprising:

one or more processors configured to communicate with a vehicle having an autonomous driving mode and a computing device of a passenger, the one or more processors configured to:

obtain vehicle status information about the vehicle when operating in the autonomous driving mode, the vehicle status information comprising information about a current trip associated with the passenger;

obtain information about a traffic signal or a traffic sign that the vehicle is approaching in the autonomous driving mode;

determine whether the traffic signal or the traffic sign will delay a pickup time for the passenger at a selected location by at least a threshold amount;

and

upon determining that the traffic signal or the traffic sign will delay the pickup time by the at least the threshold amount, cause a visual representation of the traffic signal or the traffic sign to be displayed concurrently with a map and an indication of a reason for the delay in a graphical user interface of the computing device of the passenger.

14. The computing system of claim 13 , wherein the computing system comprises the computing device of the passenger.

15. The computing system of claim 13 , wherein the computing system comprises a remote computing system in operative communication with the vehicle and the computing device of the passenger.

16. The computing system of claim 13 , wherein causation of the visual representation to be displayed in the graphical user interface includes determination of when to display the visual representation according to a distance between the vehicle and the traffic signal or the traffic sign.

17. The computing system of claim 13 , wherein the one or more processors are further configured to, upon determining that the traffic signal or the traffic sign will not delay the pickup time by the at least the threshold amount, cause the visual representation to be displayed concurrently with the map in the graphical user interface without the indication of the reason for the delay.

18. A non-transitory computer-readable medium storing instructions which, when executed, causes one or more processors of a computing device to perform a method comprising:

obtaining vehicle status information about a vehicle operating in an autonomous driving mode, the vehicle status information comprising information about a current trip associated with a passenger;

obtaining information about a traffic signal or a traffic sign that the vehicle is approaching in the autonomous driving mode;

determining whether the traffic signal or the traffic sign will delay a pickup time for the passenger at a selected location by at least a threshold amount; and

upon determining that the traffic signal or the traffic sign will delay the pickup time, causing a visual representation of the traffic signal or the traffic sign to be displayed concurrently with a map and an indication of a reason for the delay in a graphical user interface of a computing device of the passenger.

19. The non-transitory computer-readable medium of claim 18 , wherein the passenger is awaiting pickup by the vehicle.

20. The non-transitory computer-readable medium of claim 18 , wherein the indication of the reason for the delay includes textual information that explains the information about the traffic signal or the traffic sign.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 22, 2024
From: PANDIT, SALIL; PATEL, NIRMAL; HUBERT, RENAUD-ROLAND; MOON, MARIA; SCHWENDIMANN, LAUREN; VILLAR, GUILHERME; POWELL, RYAN; CRANDALL, PETER; HALL, MATTHEW COREY
To: WAYMO LLC
Reel/Frame 068966/0803 →
Continuity (3)
Continuation 18200261 · May 22, 2023
Continuation 16710113 · Dec 11, 2019
Related Publication 20250061806A1 · Feb 20, 2025
References Cited (53)
US 8676431B1 · Mariet et al. · 2014 [cited by applicant]
US 9443426B1 · Formwalt · 2016 [cited by applicant]
US 9953283B2 · Sweeney et al. · 2018 [cited by applicant]
US 10310505B1 · Hanson et al. · 2019 [cited by applicant]
US 10372130B1 · Kaushansky et al. · 2019 [cited by applicant]
US 10740729B1 · Leary · 2020 [cited by examiner]
US 10872381B1 · Leise et al. · 2020 [cited by applicant]
US 20070052586A1 · Horstemeyer · 2007 [cited by applicant]
US 20090118970A1 · Daum et al. · 2009 [cited by applicant]
US 20100117816A1 · Okada · 2010 [cited by applicant]
US 20120259706A1 · Lobaza et al. · 2012 [cited by applicant]
US 20130093890A1 · Cunningham · 2013 [cited by examiner]
US 20140227991A1 · Breton et al. · 2014 [cited by applicant]
US 20140309789A1 · Ricci · 2014 [cited by applicant]
US 20140309806A1 · Ricci · 2014 [cited by applicant]
US 20150347075A1 · Levesque et al. · 2015 [cited by applicant]
US 20160055750A1 · Linder et al. · 2016 [cited by applicant]
US 20160059864A1 · Feit et al. · 2016 [cited by applicant]
US 20160356603A1 · Hajj et al. · 2016 [cited by applicant]
US 20170096235A1 · Thomas · 2017 [cited by applicant]
US 20170293614A1 · Chen · 2017 [cited by applicant]
US 20170300049A1 · Seally · 2017 [cited by applicant]
US 20170316696A1 · Bartel · 2017 [cited by applicant]
US 20180096597A1 · Mortazavi · 2018 [cited by examiner]
US 20180126985A1 · Lee et al. · 2018 [cited by applicant]
US 20180170392A1 · Yang et al. · 2018 [cited by applicant]
US 20180261081A1 · Suzuki et al. · 2018 [cited by applicant]
US 20180334161A1 · Mizuno et al. · 2018 [cited by applicant]
US 20180365908A1 · Liu et al. · 2018 [cited by applicant]
US 20190017839A1 · Eyler et al. · 2019 [cited by applicant]
US 20190094859A1 · Nix et al. · 2019 [cited by applicant]
US 20190130745A1 · Turato · 2019 [cited by applicant]
US 20190186939A1 · Cox et al. · 2019 [cited by applicant]
US 20190265703A1 · Hicok et al. · 2019 [cited by applicant]
US 20190310627A1 · Halder · 2019 [cited by applicant]
US 20190390963A1 · Kumar et al. · 2019 [cited by applicant]
US 20200041997A1 · Tuukkanen · 2020 [cited by examiner]
US 20200092837A1 · Hartley · 2020 [cited by applicant]
US 20200139812A1 · Johnson · 2020 [cited by applicant]
US 20200257299A1 · Wang et al. · 2020 [cited by applicant]
US 20200298801A1 · Dingli · 2020 [cited by applicant]
US 20230341854A1 · Lennie et al. · 2023 [cited by applicant]
CN 108205830A · 2018 [cited by applicant]
CN 108474665A · 2018 [cited by applicant]
CN 108885773A · 2018 [cited by applicant]
CN 111429730A · 2020 [cited by examiner]
ES 2436105T3 · 2013 [cited by examiner]
JP 2014048859A · 2014 [cited by applicant]
WO 2019152471A2 · 2019 [cited by applicant]
Office Action for European Patent Application No. 20208143.6, Mar. 10, 2023. [cited by applicant]
The First Office Action for Chinese Patent Application No. 202011460417.8, Jun. 1, 2023, 18 Pages. [cited by applicant]
The Extended European Search Report for EP Application No. 20208143.6 dated Apr. 19, 2021, EPO, Munich, Germany. [cited by applicant]
The Extended European Search Report for European Patent Application No. 24198738.7, Nov. 21, 2024, 9 Pages. [cited by applicant]