IP Library Granted Patent US 12,735,069
Granted Patent B2
US 12,735,069 · App. 18/407,108 · Granted Sep 15, 2026

Robotaxi

Inventors: Gary Hicok (Mesa, AZ); Michael Cox (Menlo Park, CA); Miguel Sainz (Palo Alto, CA); Martin Hempel (Mountain View, CA); Ratin Kumar (Cupertino, CA); Timo Roman (Uusimaa, FI); Gordon Grigor (San Francisco, CA); David Nister (Bellevue, WA); Justin Ebert (Boulder, CO); Chin-Hsien Shih (Saratoga, CA); Tony Tam (Redwood City, CA); Ruchi Bhargava (Redmond, WA)
Assignee: NVIDIA Corporation
B60W60/00253G01C21/3415G01C21/3438G01C21/3617G05B13/027G05D1/0055G05D1/0242G05D1/0246G05D1/0257G05D1/228G05D1/244G05D1/247G05D1/249G05D1/617G06Q10/02G06Q50/40B60W2420/403B60W2420/408B60W2554/40B60W2555/20B60W2556/10B60W2556/40B60W2556/50G05B2219/36418G08G1/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,735,069
App. No.
18/407,108
Granted
Sep 15, 2026
Kind
B2
Abstract

A system and method for an on-demand shuttle, bus, or taxi service able to operate on private and public roads provides situational awareness and confidence displays. The shuttle may include ISO 26262 Level 4 or Level 5 functionality and can vary the route dynamically on-demand, and/or follow a predefined route or virtual rail. The shuttle is able to stop at any predetermined station along the route. The system allows passengers to request rides and interact with the system via a variety of interfaces, including without limitation a mobile device, desktop computer, or kiosks. Each shuttle preferably includes an in-vehicle controller, which preferably is an AI Supercomputer designed and optimized for autonomous vehicle functionality, with computer vision, deep learning, and real time ray tracing accelerators. An AI Dispatcher performs AI simulations to optimize system performance according to operator-specified system parameters.

Claims (40)

1 . A shuttle comprising:

a body comprising a compartment to accommodate at least one passenger;

a propulsion, drive train, and steering system to propel the body;

sensors disposed on the body including at least one light detection and ranging (LIDAR) sensor, at least one camera, and at least one global positioning system (GPS) sensor; and

a processing arrangement comprising at least one central processing unit (CPU), at least one hardware accelerator supporting at least one neural network, and at least one graphics processing unit (GPU) communicatively coupled to the sensors, the processing arrangement to:

define a virtual rail comprising a predefined route and one or more stopping points;

control the propulsion, drive train, and steering system to propel the body along at least one of the virtual rail or a deviation from the virtual rail, the deviation from the virtual rail being used when one or more conditions are satisfied;

identify passengers to ride in the shuttle; and

based at least in part on situational awareness provided by the at least one neural network, autonomously determining whether to selectively stop the shuttle at individual stopping points of the one or more stopping points of the virtual rail to at least one of take on or discharge passengers.

2 . The shuttle of claim 1 , wherein the one or more conditions include at least one of changing environment conditions or dynamic detection of objects.

3 . The shuttle of claim 1 , wherein the sensors provide sensor data to the processing arrangement, and the processing arrangement analyzes the sensor data to monitor locations at least one of in front of the body, behind the body, to the left of the body, to the right of the body, above the body, or below the body.

4 . The shuttle of claim 1 , wherein the processing arrangement is further to generate the virtual rail based at least on stored, previous routes the shuttle has followed in the past.

5 . The shuttle of claim 1 , wherein the processing arrangement includes a machine learning component trained on the virtual rail by a human driver and/or which receives information concerning the virtual rail from another vehicle or other source.

6 . The shuttle of claim 1 , wherein the processing arrangement is to use discovery and a high-definition dynamic mapping process to map the virtual rail.

7 . The shuttle of claim 1 , wherein the processing arrangement is to use sensor data obtained using the sensors to survey an environment and update the virtual rail based at least on detected changes in the environment.

8 . The shuttle of claim 1 , wherein the processing arrangement is further to use sensor data obtained using the sensors to detect dynamic objects including parked cars, pedestrians, and animals for avoidance while following the virtual rail.

9 . The shuttle of claim 1 , wherein the processing arrangement is to stop the shuttle at individual stopping points of the one or more stopping points along the virtual rail in response to passenger or prospective passenger requests.

10 . The shuttle of claim 1 , wherein the processing arrangement includes a security layer and a hardware computer vision accelerator.

11 . The shuttle of claim 1 , further comprising a prospective passenger signaling display disposed on an exterior surface of the body, the signaling display selectively indicating the shuttle is waiting for a prospective passenger to board or depart.

12 . The shuttle of claim 1 , wherein the shuttle comprises a van, a bus, a robo-taxi, a sedan, a limousine, or any vehicle able to be adapted for on-demand transportation or ride-sharing service.

13 . The shuttle of claim 1 wherein the processing arrangement complies with at least an ASIL C functional safety level of the ISO 26262 functional safety specification.

14 . A method of operating a shuttle including performing, using at least one central processing unit (CPU), at least one hardware accelerator supporting at least one neural network, and at least one graphics processing unit (GPU) on the shuttle and communicatively coupled to sensors, operations comprising:

defining a virtual rail comprising a predefined route and one or more stopping points;

sensing in an environment of the shuttle using the sensors including at least one light detection and ranging (LIDAR) sensor, at least one camera, and at least one global positioning system (GPS) sensor;

in response to the sensing, controlling propulsion, drive train, and steering systems of the shuttle to propel the shuttle along at least one of the virtual rail or a deviation from the virtual rail, the deviation from the virtual rail being used when one or more conditions are satisfied;

identifying passengers to ride in the shuttle; and

based at least in part on situational awareness provided by the at least one neural network, autonomously determining whether to selectively stop the shuttle at individual stopping points of the one or more stopping points of the virtual rail to at least one of take on or discharge passengers.

15 . The method of claim 14 wherein the one or more conditions include at least one of changing environment conditions or dynamic detection of objects.

16 . The method of claim 14 , further comprising using sensor data obtained using the sensors to update the virtual rail based at least on detected changes in the environment.

17 . The method of claim 14 , further comprising stopping the shuttle at individual stopping points of the one or more stopping points along the virtual rail in response to passenger or prospective passenger requests.

18 . The method of claim 14 wherein the operations comply with at least an ASIL C functional safety level of the ISO 26262 functional safety specification.

19 . In a shuttle comprising a propulsion, drive train, and steering system to propel the shuttle, and sensors disposed on the shuttle including at least one light detection and ranging (LIDAR) sensor, at least one camera, and at least one global positioning system (GPS) sensor, a control system on the shuttle comprising at least one central processing unit (CPU), at least one hardware accelerator supporting at least one neural network, and at least one graphics processing unit (GPU) communicatively coupled to the sensors, the control system on the shuttle performing operations comprising:

defining a virtual rail comprising one or more stopping points;

controlling the propulsion, drive train, and steering system to propel the shuttle along at least one of the virtual rail or a deviation from the virtual rail, the deviation from the virtual rail being used when one or more conditions are satisfied;

identify passengers to ride in the shuttle; and

based at least in part on situational awareness provided by the at least one neural network, autonomously determining whether to selectively stop the shuttle at individual stopping points of the one or more stopping points of the virtual rail to at least one of take on or discharge passengers.

20 . The control system of claim 19 , wherein the one or more conditions include at least one of changing environment conditions or dynamic detection of objects.

21 . The control system of claim 19 , wherein the at least one hardware accelerator uses sensor data obtained using the sensors to survey an environment and update the virtual rail based at least on detected changes in the environment.

22 . The control system of claim 19 , wherein the shuttle comprises a van, a bus, a robo-taxi, a sedan, a limousine, or any vehicle able to be adapted for on-demand transportation or ride-sharing service.

23 . The control system of claim 19 wherein the control system complies with at least an ASIL C functional safety level of the ISO 26262 functional safety specification.

Continuity (4)
Division 17896825 · Aug 26, 2022
Division 16286330 · Feb 26, 2019
Provisional Application 62635503 · Feb 26, 2018
Related Publication 20250222958A1 · Jul 10, 2025
References Cited (82)
US 8436862B2 · Yu · 2013 [cited by applicant]
US 9188985B1 · Hobbs et al. · 2015 [cited by applicant]
US 9811086B1 · Poeppel · 2017 [cited by applicant]
US 9904375B1 · Donnelly · 2018 [cited by applicant]
US 10346888B2 · Nix · 2019 [cited by applicant]
US 10382862B2 · Smith · 2019 [cited by applicant]
US 10429846B2 · Nix · 2019 [cited by applicant]
US 10452974B1 · Casie · 2019 [cited by applicant]
US 10466698B1 · Valasek · 2019 [cited by applicant]
US 10665140B1 · Ahn · 2020 [cited by applicant]
US 10768621B1 · Nix · 2020 [cited by examiner]
US 10803089B1 · Seibert · 2020 [cited by applicant]
US 20080009964A1 · Bruemmer et al. · 2008 [cited by applicant]
US 20080009965A1 · Bruemmer et al. · 2008 [cited by applicant]
US 20110161616A1 · Tarjan · 2011 [cited by applicant]
US 20150336502A1 · Hillis · 2015 [cited by applicant]
US 20160171515A1 · Radhakrishnan · 2016 [cited by applicant]
US 20160247095A1 · Scicluna · 2016 [cited by examiner]
US 20160321074A1 · Hung · 2016 [cited by applicant]
US 20160370194A1 · Colijn et al. · 2016 [cited by applicant]
US 20170169366A1 · Klein · 2017 [cited by applicant]
US 20170169535A1 · Tolkin · 2017 [cited by applicant]
US 20170193627A1 · Urmson et al. · 2017 [cited by applicant]
US 20170277181A1 · Fairfield et al. · 2017 [cited by applicant]
US 20170277191A1 · Fairfield et al. · 2017 [cited by applicant]
US 20170316533A1 · Goldman-Shenhar et al. · 2017 [cited by applicant]
US 20170352125A1 · Dicker · 2017 [cited by applicant]
US 20170370734A1 · Colijn et al. · 2017 [cited by applicant]
US 20180052000A1 · Larner et al. · 2018 [cited by applicant]
US 20180059688A1 · Benraz · 2018 [cited by examiner]
US 20180060459A1 · English · 2018 [cited by applicant]
US 20180074495A1 · Myers · 2018 [cited by examiner]
US 20180093663A1 · Kim · 2018 [cited by applicant]
US 20180107222A1 · Fairfield et al. · 2018 [cited by applicant]
US 20180136651A1 · Levinson · 2018 [cited by examiner]
US 20180136655A1 · Kim · 2018 [cited by applicant]
US 20180164809A1 · Moosaei · 2018 [cited by examiner]
US 20180201273A1 · Xiao · 2018 [cited by applicant]
US 20180224853A1 · Izhikevich · 2018 [cited by examiner]
US 20180238698A1 · Pedersen · 2018 [cited by applicant]
US 20180300964A1 · Lakshamanan · 2018 [cited by applicant]
US 20180338229A1 · Nemec et al. · 2018 [cited by applicant]
US 20180339712A1 · Kislovskiy · 2018 [cited by applicant]
US 20190005606A1 · Yang · 2019 [cited by examiner]
US 20190018411A1 · Herbach · 2019 [cited by applicant]
US 20190080514A1 · Nasi · 2019 [cited by applicant]
US 20190137290A1 · Levy · 2019 [cited by applicant]
US 20190156254A1 · Hansen · 2019 [cited by examiner]
US 20190187705A1 · Ganguli · 2019 [cited by applicant]
US 20190197430A1 · Arditi · 2019 [cited by examiner]
US 20190236322A1 · Arquero · 2019 [cited by applicant]
US 20190263422A1 · Enthaler · 2019 [cited by applicant]
US 20200167697A1 · Tran · 2020 [cited by applicant]
US 20200394393A1 · Kraft · 2020 [cited by examiner]
US 20210088341A1 · Macneille · 2021 [cited by examiner]
US 20210125226A1 · Macneille · 2021 [cited by examiner]
US 20210188088A1 · Kuehne · 2021 [cited by applicant]
CN 101332820A · 2008 [cited by applicant]
CN 108973898A · 2018 [cited by applicant]
DE 102015122212B3 · 2017 [cited by examiner]
JP H0865804A · 1996 [cited by applicant]
KR 20210109410A · 2021 [cited by applicant]
WO 2015041595A1 · 2015 [cited by applicant]
WO 2019151955A1 · 2019 [cited by applicant]
Bonnay et al., “Fuzzy Lane-Track Control for the Automatic Guidance of an Automotive Vehicle”, IFAC Advances in Automotive Control, pp. 35-40, 2001. [cited by applicant]
Zalila et al., “Lateral Guidance of an Autonomous Vehicle by a Fuzzy Logic Controller”, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No. 98CH36218), San Diego,… [cited by applicant]
International Search Report and Written Opinion of the International Searching Authority mailed Jun. 28, 2019, issued in International Application No. PCT/US2019/019635, 21 pages. [cited by applicant]
Ackerman, Evan, “Hail, Robo-taxi!”, Top Tech 2017, IEEE Spectrum, Jan. 2017, vol. 54, No. 1, pp. 26-29, XP11638858. [cited by applicant]
Bojarski, Mariusz, et al., “Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car,” Cornell University Library, Apr. 25, 2017, pp. 1-8, XP80765549. [cited by applicant]
Howard, Dwight, “Enhanced Driver Safety with Advanced Vision Systems,” 2018 Pan Pacific Microelectronics Symposium, Feb. 5, 2018, 7 pages, XP33331614. [cited by applicant]
Moras, Julien, et al., “Drivable Space Characterization using Automotive Lidar and Georeferenced Map Information,” 2012 Intelligent Vehicles Symposium, Jun. 3, 2012, pp. 778-783, XP32453043. [cited by applicant]
Mujtaba, Hassan, “NVIDIA Drive Xavier SOC Detailed—A Marvel of Engineering, Biggest and Most Complex SOC Design To Date With 9 Billion Transistors,” Jan. 8, 2018, 10 pages, https://wccftech.com/nvidia-drive-xavier-soc-d… [cited by applicant]
Chen, “Engineering Uber's Self-Driving Car Visualization Platform for the Web,” Aug. 28, 2017, https://eng.uber.com/atg-dataviz/ (Year: 2017). [cited by applicant]
National Transportation Safety Board, Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian Tempe, Arizona, NTSB/HAR-19/03 PB2019-101402 (NTSB: Mar. 18, 2018) https://www.ntsb.gov… [cited by applicant]
Waymo, On the Road to Fully Self-Driving: Waymo Safety Report (Waymo: 2017) https://docs.huihoo.com/car/Waymo-Safety-Report-2017-1 0.pdf. (Year: 2017). [cited by applicant]
Courtney Linder, “Say goodbye to these Uber self-driving cars. But don't worry, there's a fresh fleet coming,” Pittsburgh Post-Gazette, Sep. 20, 2017. https://www.post-gazette.com/business/tech-news/2017/09/20/uber-atg-… [cited by applicant]
Huang et al. “Adas on Cots with OpenCL: A Case Study with Lane Detection,” IEEE Transactions on Computers 67, No. 4, Apr. 1, 2018 (published Oct. 4, 2017), https://ieeexplore.ieee.org/abstract/document/8057795. (Year: 2… [cited by applicant]
Marcher et al. “Automotive Embedded Software: Mitigation Challenges to Multi-Core Computing Platforms,” (2015) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7281937 (Year: 2015). [cited by applicant]
Lee et al., “VADI: GPU Virtualization for an Automotive Platform,” IEEE Transactions on Industrial Informatics 12, No. 1 (Feb. 2016), https://ieeexplore.ieee.org/document/7359144 (Year: 2016). [cited by applicant]
Lari et al. “Techniques for an on-demand structural redundancy for massively parallel processor arrays,” Journal of Systems Architecture 61, No. 10 (Nov. 2015): 615-627, https://www.sciencedirect.com/science/article/pii… [cited by applicant]
En.wikipedia.org s.v. “massively parallel,” accessed Dec. 13, 2022, https://en.wikipedia.org/wiki/Massively_parallel (Year: 2022). [cited by applicant]
Eda Kavlakoglu, “AI vs. Machine Learning vs. Deep Learning vs. Neural Networks: What's the Difference?,” published on May 27, 2020 (https://www.ibm.com/cloud/blog/ai-vs-machine-learning-vs-deep-learning-vs-neural-networ… [cited by applicant]