IP Library Granted Patent US 12,488,291
Granted Patent B2
US 12,488,291 · App. 18/434,268 · Granted Dec 2, 2025

Context-based remote autonomous vehicle assistance

Inventors: Sean Shanshi Chen (San Francisco, CA); Samann Ghorbanian-Matloob (San Francisco, CA); Brent Justin Goldman (San Francisco, CA); Michael Guanran Huang (San Francisco, CA)
Assignee: Uber Technologies, Inc.
G06Q10/02G05D1/0088G05D1/0212G05D1/0276G05D1/228G05D1/247G05D1/646G06Q10/06315G06Q50/40H04W4/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,488,291
App. No.
18/434,268
Granted
Dec 2, 2025
Kind
B2
Abstract

Systems and methods for controlling autonomous vehicles are provided. Assisted autonomy tasks facilitated by operators for a plurality of autonomous vehicles can be tracked in order to generate operator attributes for each of a plurality of operators. The attributes for an operator can be based on tracking one or more respective assisted autonomy tasks facilitated by the operator. The operator attributes can be used to facilitate enhanced remote operations for autonomous vehicles. For example, request parameters can be obtained in response to a request for remote assistance associated with an autonomous vehicle. An operator can be selected to assist with autonomy tasks for the autonomous vehicle based at least in part on the operator attributes for the operator and the request parameters associated with the request. Remote assistance for the first autonomous vehicle can be initiated, facilitated by the first operator in response to the request for remote assistance.

Claims (44)

1 . A computing system comprising:

one or more processors; and

one or more tangible, non-transitory, computer-readable media that collectively store instructions that when executed by the one or more processors cause the computing system to perform operations comprising:

receiving, via a request interface of the computing system, a communication from a user computing device associated with a request for assistance with an autonomous vehicle service, the request interface configured to communicate with a plurality of user computing devices including the user computing device;

determining an autonomous vehicle corresponding to the autonomous vehicle service based at least in part on the communication from the user computing device;

in response to receiving the request for assistance with the autonomous vehicle service, generating assistance data; and

causing the autonomous vehicle to perform an action based at least on the assistance data.

2 . The computing system of claim 1 , wherein the determining the autonomous vehicle is further based on an identifier associated with the user computing device.

3 . The computing system of claim 1 , wherein the determining the autonomous vehicle is further based on a service record of the autonomous vehicle.

4 . The computing system of claim 1 , wherein the causing the autonomous vehicle to perform the action comprises causing an onboard computing device of the autonomous vehicle to generate one or more graphical user interfaces for display.

5 . The computing system of claim 1 , wherein the operations further comprise:

prompting the user computing device to provide identifying data describing the autonomous vehicle, the determining of the autonomous vehicle being based on the identifying data.

6 . The computing system of claim 1 , wherein the communication from the user computing device indicates at least one of a failure of a component of the autonomous vehicle or a medical problem of a passenger of the autonomous vehicle.

7 . The computing system of claim 1 , wherein the operations further comprise:

accessing a respective plurality of operator attributes describing a plurality of operators; and

selecting a first operator of the plurality of operators to assist the autonomous vehicle in response to the communication from the user computing device, the selecting based at least in part on the plurality of operator attributes.

8 . The computing system of claim 1 , wherein the causing the autonomous vehicle to perform the action is based at least in part on data indicative of at least one of a location of the autonomous vehicle, an identification of the autonomous vehicle, a type of the autonomous vehicle, a vehicle capability of the autonomous vehicle, or an assistance type associated with the request for assistance with the autonomous vehicle service.

9 . The computing system of claim 1 , wherein the operations further comprise:

obtaining data associated with a target vehicle path for the autonomous vehicle, wherein the causing the autonomous vehicle to perform the action is based at least in part on the data associated with the target vehicle path.

10 . The computing system of claim 1 , wherein the operations further comprise:

obtaining object classification data describing an unclassified object at a geographic area associated with the autonomous vehicle, wherein the causing the autonomous vehicle to perform the action is based at least in part on the object classification data.

11 . A computer-implemented method, comprising:

receiving, via a request interface of a computing system, a communication from a user computing device associated with a request for assistance with an autonomous vehicle service, the request interface configured to communicate with a plurality of user computing devices including the user computing device;

determining, by the computing system, an autonomous vehicle corresponding to the autonomous vehicle service based at least in part on the communication from the user computing device;

in response to receiving the request for assistance with the autonomous vehicle service, generating, by the computing system, assistance data; and

causing the autonomous vehicle to perform an action based at least on the assistance data.

12 . The method of claim 11 , wherein the determining the autonomous vehicle is further based on an identifier associated with the user computing device or on a service record of the autonomous vehicle.

13 . The method of claim 11 , wherein the causing the autonomous vehicle to perform the action comprises causing an onboard computing device of the autonomous vehicle to generate one or more graphical user interfaces for display.

14 . The method of claim 11 , further comprising:

prompting the user computing device to provide identifying data describing the autonomous vehicle, the determining of the autonomous vehicle being based on the identifying data.

15 . The method of claim 11 , wherein the communication from the user computing device indicates at least one of a failure of a component of the autonomous vehicle or a medical problem of a passenger of the autonomous vehicle.

16 . The method of claim 11 , further comprising:

accessing a respective plurality of operator attributes describing a plurality of operators; and

selecting a first operator of the plurality of operators to assist the autonomous vehicle in response to the communication from the user computing device, the selecting based at least in part on the plurality of operator attributes.

17 . The method of claim 11 , wherein the causing the autonomous vehicle to perform the action is based at least in part on data indicative of at least one of a location of the autonomous vehicle, an identification of the autonomous vehicle, a type of the autonomous vehicle, a vehicle capability of the autonomous vehicle, or an assistance type associated with the request for assistance with the autonomous vehicle service.

18 . The method of claim 11 , further comprising:

obtaining data associated with a target vehicle path for the autonomous vehicle, wherein the causing the autonomous vehicle to perform the action is based at least in part on the data associated with the target vehicle path.

19 . The method of claim 11 , further comprising:

obtaining object classification data describing an unclassified object at a geographic area associated with the autonomous vehicle, wherein the causing the autonomous vehicle to perform the action is based at least in part on the object classification data.

20 . A non-transitory machine-readable medium comprising instructions thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving, via a request interface of a computing system, a communication from a user computing device associated with a request for assistance with an autonomous vehicle service, the request interface configured to communicate with a plurality of user computing devices including the user computing device;

determining an autonomous vehicle corresponding to the autonomous vehicle service based at least in part on the communication from the user computing device;

in response to receiving the request for assistance with the autonomous vehicle service, generating assistance data; and

causing the autonomous vehicle to perform an action based at least on the assistance data.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 11, 2024
From: CHEN, SEAN SHANSHI; GHORBANIAN-MATLOOB, SAMANN; GOLDMAN, BRENT JUSTIN; HUANG, MICHAEL GUANRAN
To: UATC, LLC
Reel/Frame 067075/0395 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 11, 2024
From: UATC, LLC
To: UBER TECHNOLOGIES, INC.
Reel/Frame 067097/0927 →
Continuity (4)
Continuation 18056695 · Nov 17, 2022
Continuation 16573560 · Sep 17, 2019
Provisional Application 62811168 · Feb 27, 2019
Related Publication 20240211818A1 · Jun 27, 2024
References Cited (72)
US 4405943A · Kanaly · 1983 [cited by applicant]
US 6405132B1 · Breed et al. · 2002 [cited by applicant]
US 7359004B2 · Yu et al. · 2008 [cited by applicant]
US 8370392B1 · Hunt · 2013 [cited by applicant]
US 8601595B2 · Gelvin et al. · 2013 [cited by applicant]
US 9384402B1 · Furman et al. · 2016 [cited by applicant]
US 9507346B1 · Levinson et al. · 2016 [cited by applicant]
US 9612123B1 · Levinson et al. · 2017 [cited by applicant]
US 9668108B2 · Li et al. · 2017 [cited by applicant]
US 10007263B1 · Fields et al. · 2018 [cited by applicant]
US 10156848B1 · Konrardy · 2018 [cited by examiner]
US 10328897B1 · Nabbe et al. · 2019 [cited by applicant]
US 10692377B1 · Kentley-klay · 2020 [cited by applicant]
US 10955851B2 · Ghafarianzadeh et al. · 2021 [cited by applicant]
US 11526816B2 · Chen et al. · 2022 [cited by applicant]
US 11922341B2 · Chen et al. · 2024 [cited by applicant]
US 20070185646A1 · Neugebauer et al. · 2007 [cited by applicant]
US 20110208417A1 · Fink et al. · 2011 [cited by applicant]
US 20150178953A1 · Gao et al. · 2015 [cited by applicant]
US 20150244826A1 · Stenneth · 2015 [cited by applicant]
US 20150248131A1 · Fairfield et al. · 2015 [cited by applicant]
US 20150339928A1 · Ramanujam · 2015 [cited by examiner]
US 20160139594A1 · Okumura et al. · 2016 [cited by applicant]
US 20160321674A1 · Lux · 2016 [cited by applicant]
US 20160358475A1 · Prokhorov · 2016 [cited by applicant]
US 20170147951A1 · Meyer · 2017 [cited by examiner]
US 20170169366A1 · Klein · 2017 [cited by examiner]
US 20170192423A1 · Rust et al. · 2017 [cited by applicant]
US 20170192426A1 · Rust · 2017 [cited by applicant]
US 20170223712A1 · Stephens et al. · 2017 [cited by applicant]
US 20170285637A1 · Salinger et al. · 2017 [cited by applicant]
US 20170308082A1 · Ullrich et al. · 2017 [cited by applicant]
US 20180011485A1 · Ferren · 2018 [cited by applicant]
US 20180114258A1 · Ross et al. · 2018 [cited by applicant]
US 20180181118A1 · Yoneda et al. · 2018 [cited by applicant]
US 20180196444A1 · Yi et al. · 2018 [cited by applicant]
US 20180257661A1 · Kroop et al. · 2018 [cited by applicant]
US 20180281815A1 · Stentz · 2018 [cited by applicant]
US 20180356814A1 · Brooks · 2018 [cited by applicant]
US 20200050190A1 · Patel et al. · 2020 [cited by applicant]
US 20200184591A1 · Balu et al. · 2020 [cited by applicant]
US 20200272949A1 · Chen et al. · 2020 [cited by applicant]
US 20200278674A1 · Ferren · 2020 [cited by applicant]
US 20230082760A1 · Chen et al. · 2023 [cited by applicant]
EP 3931773 · 2022 [cited by applicant]
IN 202127038567 · 2021 [cited by applicant]
WO WO2017040260A1 · 2017 [cited by examiner]
WO 2017079222 · 2017 [cited by applicant]
WO 2019106318 · 2019 [cited by applicant]
WO 2020176474 · 2020 [cited by applicant]
“International Application Serial No. PCT US2020 019660, International Search Report mailed Jul. 6, 2020”, 5 pgs. [cited by applicant]
“International Application Serial No. PCT US2020 019660, Written Opinion mailed Jul. 6, 2020”, 9 pgs. [cited by applicant]
“International Application Serial No. PCT US2020 019660, International Preliminary Report on Patentability mailed Sep. 10, 2021”, 21 pgs. [cited by applicant]
“International Application Serial No. PCT US2020 019660, Invitation to Pay Additional Fees mailed May 13, 2020”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 16/573,560, Non Final Office Action mailed Mar. 25, 2022”, 9 pgs. [cited by applicant]
“European Application Serial No. 20714059.1, Response to Communication persuant to Rules 161 and 162 filed Apr. 19, 2022”, 29 pgs. [cited by applicant]
“U.S. Appl. No. 16/573,560, Response filed Jun. 27, 2022 to Non Final Office Action mailed Mar. 25, 2022”, 13 pgs. [cited by applicant]
“U.S. Appl. No. 16/573,560, Notice of Allowance mailed Jul. 20, 2022”, 7 pgs. [cited by applicant]
“U.S. Appl. No. 16/573,560, 312 Amendment filed Oct. 20, 2022”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 16/573,560, PTO Response to Rule 312 Communication mailed Oct. 28, 2022”, 2 pgs. [cited by applicant]
“Indian Application Serial No. 202127038567, First Examination Report mailed Feb. 15, 2023”, 6 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,695, Non Final Office Action mailed May 2, 2023”, 14 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,695, Response filed Aug. 2, 2023 to Non Final Office Action mailed May 2, 2023”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,695, Final Office Action mailed Aug. 18, 2023”, 5 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,695, Response filed Oct. 16, 2023 to Final Office Action mailed Aug. 18, 2023”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 18/056,695, Notice of Allowance mailed Nov. 1, 2023”, 9 pgs. [cited by applicant]
“Indian Application Serial No. 202127038567, Response filed Nov. 15, 2023 to First Examination Report mailed Feb. 15, 2023”, w English claims, 33 pgs. [cited by applicant]
Fong, Terrence, “Vehicle teleoperation interfaces”, Autonomous Robots 11, (2001), 10 pgs. [cited by applicant]
Reuschenbach, Arturo, “iDriver-human machine interface for autonomous cars”, Eighth International Conference on Infom1ation Technology: New Generations, Las Vegas, NV, (2011), 6 pgs. [cited by applicant]
U.S. Appl. No. 16/573,560 U.S. Pat. No. 11,526,816, filed Sep. 17, 2019, Context-Based Remote Autonomous Vehicle Assistance. [cited by applicant]
U.S. Appl. No. 18/056,695 U.S. Pat. No. 11,922,341, filed Nov. 17, 2022, Context-Based Remote Autonomous Vehicle Assistance. [cited by applicant]
“European Application Serial No. 20714059.1, Communication Pursuant to Article 94(3) EPC mailed Jun. 27, 2025”, 8 pgs. [cited by applicant]