IP Library Granted Patent US 12,315,315
Granted Patent B2
US 12,315,315 · App. 18/403,415 · Granted May 27, 2025

Systems and methods to obtain feedback in response to autonomous vehicle failure events

Inventors: Molly Castle Nix (San Francisco, CA); Sean Chin (Pittsburgh, PA); Dennis Zhao (Pittsburgh, PA)
Assignee: AURORA OPERATIONS, INC.
G07C5/0841B60K35/00B60W50/14G01C21/3697G07C5/0825B60K35/10B60K35/20B60K35/28B60K35/29B60K35/80B60K35/81B60K2360/115B60K2360/119B60K2360/1438B60K2360/175B60K2360/186B60K2360/566B60W2050/146B60W2540/215G05D1/021G05D1/69
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,315,315
App. No.
18/403,415
Granted
May 27, 2025
Kind
B2
Abstract

The present disclosure provides systems and methods to obtain feedback descriptive of autonomous vehicle failures. In particular, the systems and methods of the present disclosure can detect that a vehicle failure event occurred at an autonomous vehicle and, in response, provide an interactive user interface that enables a human located within the autonomous vehicle to enter feedback that describes the vehicle failure event. Thus, the systems and methods of the present disclosure can actively prompt and/or enable entry of feedback in response to a particular instance of a vehicle failure event, thereby enabling improved and streamlined collection of information about autonomous vehicle failures.

Claims (43)

1. An autonomous vehicle computing system for an autonomous vehicle, comprising:

a microphone;

one or more processors; and

one or more non-transitory computer-readable media that store instructions that are executable by the one or more processors to cause the computing system to perform operations, the operations comprising:

determining a human intervention event in which a human driver assumed control of the autonomous vehicle;

outputting, using an interface device and responsive to determining the human intervention event, a prompt for the human driver to provide feedback;

obtaining, using the microphone, a voice response that includes feedback regarding autonomous driving behavior of the vehicle; and

associating the voice response with data collected by the autonomous vehicle over a time window prior to the human intervention event.

2. The autonomous vehicle computing system of claim 1 , wherein outputting the prompt comprises rendering the prompt on a display device.

3. The autonomous vehicle computing system of claim 1 , wherein outputting the prompt comprises rendering a prompt on a display device while also rendering navigational information in a navigational portion of the user interface.

4. The autonomous vehicle computing system of claim 2 , the operations comprising:

determining that a predefined period has elapsed after rendering the prompt; and

dismissing the prompt.

5. The autonomous vehicle computing system of claim 1 , wherein outputting the prompt comprises rendering an audio-based prompt.

6. The autonomous vehicle computing system of claim 5 , wherein rendering the audio-based prompt comprises initiating a voice conversation using a personal assistant system.

7. The autonomous vehicle computing system of claim 1 , the operations comprising:

transmitting the voice response and the data collected by the autonomous vehicle to a training computing system for training a machine-learned model.

8. One or more non-transitory computer-readable media that store instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising:

determining a human intervention event in which a human driver assumed control of the autonomous vehicle;

outputting, using an interface device and responsive to determining the human intervention event, a prompt for the human driver to provide feedback;

obtaining, using a microphone, a voice response that includes feedback regarding autonomous driving behavior of the vehicle; and

associating the voice response with data collected by the autonomous vehicle over a time window prior to the human intervention event.

9. The one or more non-transitory computer-readable media of claim 8 , wherein outputting the prompt comprises rendering the prompt on a display device.

10. The one or more non-transitory computer-readable media of claim 8 , wherein outputting the prompt comprises rendering a prompt on a display device while also rendering navigational information in a navigational portion of the user interface.

11. The one or more non-transitory computer-readable media of claim 9 , the operations comprising:

determining that a predefined period has elapsed after rendering the prompt; and

dismissing the prompt.

12. The one or more non-transitory computer-readable media of claim 8 , wherein outputting the prompt comprises rendering an audio-based prompt.

13. The one or more non-transitory computer-readable media of claim 12 , wherein rendering the audio-based prompt comprises initiating a voice conversation using a personal assistant system.

14. The one or more non-transitory computer-readable media of claim 8 , the operations comprising:

transmitting the voice response and the data collected by the autonomous vehicle to a training computing system for training a machine-learned model.

15. A computer-implemented method, comprising:

determining a human intervention event in which a human driver assumed control of the autonomous vehicle;

outputting, using an interface device and responsive to determining the human intervention event, a prompt for the human driver to provide feedback;

obtaining, using a microphone, a voice response that includes feedback regarding autonomous driving behavior of the vehicle; and

associating the voice response with data collected by the autonomous vehicle over a time window prior to the human intervention event.

16. The method of claim 15 , wherein outputting the prompt comprises rendering the prompt on a display device.

17. The method of claim 15 , wherein outputting the prompt comprises rendering a prompt on a display device while also rendering navigational information in a navigational portion of the user interface.

18. The method of claim 17 , the operations comprising:

determining that a predefined period has elapsed after rendering the prompt; and

dismissing the prompt.

19. The method of claim 15 , wherein outputting the prompt comprises rendering an audio-based prompt.

20. The method of claim 15 , wherein rendering the audio-based prompt comprises initiating a voice conversation using a personal assistant system.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2024
From: UATC, LLC
To: AURORA OPERATIONS, INC.
Reel/Frame 067733/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 11, 2024
From: UBER TECHNOLOGIES, INC.
To: UATC, LLC
Reel/Frame 067077/0243 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 3, 2024
From: NIX, MOLLY CASTLE; CHIN, SEAN RHEE; ZHAO, DENNIS
To: UBER TECHNOLOGIES, INC.
Reel/Frame 066011/0498 →
Continuity (4)
Continuation 18154130 · Jan 13, 2023
Continuation 16741282 · Jan 13, 2020
Continuation 15625255 · Jun 16, 2017
Related Publication 20240144748A1 · May 2, 2024
References Cited (45)
US 6314422B1 · Barket et al. · 2001 [cited by applicant]
US 6556971B1 · Rigsby · 2003 [cited by examiner]
US 9183681B2 · Fish · 2015 [cited by applicant]
US 9299197B2 · Fish · 2016 [cited by applicant]
US 9335764B2 · Herz · 2016 [cited by examiner]
US 9342935B2 · Lundsgaard · 2016 [cited by applicant]
US 9346400B2 · Attard · 2016 [cited by applicant]
US 9406177B2 · Attard · 2016 [cited by applicant]
US 9432347B2 · Thoppae · 2016 [cited by applicant]
US 9791860B2 · Phillips · 2017 [cited by applicant]
US 9858731B2 · Fish · 2018 [cited by applicant]
US 10156960B2 · Marshall · 2018 [cited by applicant]
US 10559140B2 · Nix et al. · 2020 [cited by applicant]
US 10699404B1 · Nussbaum · 2020 [cited by applicant]
US 11568689B2 · Nix et al. · 2023 [cited by applicant]
US 20060122748A1 · Nou · 2006 [cited by applicant]
US 20080161988A1 · Oesterling · 2008 [cited by examiner]
US 20090167515A1 · Scherzinger et al. · 2009 [cited by applicant]
US 20100228429A1 · Lowrey et al. · 2010 [cited by applicant]
US 20110191722A1 · Gill et al. · 2011 [cited by applicant]
US 20110209074A1 · Gill et al. · 2011 [cited by applicant]
US 20120259504A1 · Lowrey et al. · 2012 [cited by applicant]
US 20140129077A1 · Selkirk et al. · 2014 [cited by applicant]
US 20140195100A1 · Lundsgaard et al. · 2014 [cited by applicant]
US 20140277908A1 · Fish et al. · 2014 [cited by applicant]
US 20140282255A1 · Fish et al. · 2014 [cited by applicant]
US 20140357295A1 · Skomra · 2014 [cited by examiner]
US 20150088358A1 · Yopp · 2015 [cited by applicant]
US 20150121275A1 · Marshall et al. · 2015 [cited by applicant]
US 20150187146A1 · Chen et al. · 2015 [cited by applicant]
US 20150346722A1 · Herz · 2015 [cited by examiner]
US 20170046048A1 · Marshall et al. · 2017 [cited by applicant]
US 20170337027A1 · Chan · 2017 [cited by examiner]
US 20170364869A1 · Tarte · 2017 [cited by examiner]
US 20180364711A1 · Goldfain et al. · 2018 [cited by applicant]
US 20190172278A1 · Castro Duran et al. · 2019 [cited by applicant]
US 20200151973A1 · Nix et al. · 2020 [cited by applicant]
AU 2017100444 · 2017 [cited by applicant]
DE 102014013960 · 2016 [cited by applicant]
DE 102015007242 · 2016 [cited by applicant]
DE 102015210428 · 2016 [cited by applicant]
WO WO2016109540 · 2016 [cited by applicant]
Zurich Distriouted by Harbor Freight Tools (Year: 2018). [cited by examiner]
PCT International Search Report and Written Opinion for PCT/US2018/037757, mailed on Oct. 8, 2018, 13 pages. [cited by applicant]
Zurich manual, distributed by Harbor Freight Tools, 2018. [cited by applicant]