IP Library Granted Patent US 12,277,522
Granted Patent B2
US 12,277,522 · App. 17/650,288 · Granted Apr 15, 2025

System and method for generating a planned path using a phantom vehicle

Inventors: Shubhashree Venkatesh (Fremont, CA); Noe Brito (Cupertino, CA); Yee-Ning Cheng (Sunnyvale, CA); Madhav Chhura (Whittier, CA); Sebastian Dovenor (Pittsburgh, PA); John Drake (Cranbury, NJ); Jonathan Pan (Campbell, CA); Jason Parraga (Fremont, CA); Scott Plant (San Jose, CA)
Assignee: Volkswagen Group of America Investments, LLC
G06Q10/083B60W50/0097B60W60/0011B60W60/00253B60W60/00256G01C21/36G01C21/3841G01C21/3856G01C21/387G06F16/285G06Q10/06311G08G1/20H04L9/3213H04L63/0807H04L63/0823H04L63/101H04L63/102H04L63/105H04L63/107B60W2552/00B60W2554/00B60W2556/40B60W2556/45
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,277,522
App. No.
17/650,288
Granted
Apr 15, 2025
Kind
B2
Abstract

Methods and systems that use a phantom vehicle to help generate a planned path for a real-world vehicle are described. The system will identify a starting point and a destination for a trip of the real-world vehicle. The system will select, from the data store of vehicle profiles, a phantom vehicle having an associated motion planning system that corresponds to a system that is deployed on the real-world vehicle. The system will use a high definition map to generate a planned route for the real-world vehicle from the starting point to the destination in the map. The system will run a simulation in which the phantom vehicle moves along the planned route in the map. The system will then output a record of the simulation to a user of the real world-vehicle or to a system of the real-world vehicle.

Claims (81)

1. A system for generating a planned path for a vehicle, the system comprising:

a cloud service provider processor, and programming instructions that are configured to cause the cloud service provider processor to:

receive a trip request;

generate a planned route for a real-world vehicle in response to the trip request;

simulate a trip by a virtual vehicle by:

selecting the virtual vehicle from a plurality of different virtual vehicles profiled in a data store by characteristics of real-world vehicles, the virtual vehicle that is selected has an associated motion planning system that corresponds to a motion planning system that is deployed on the real-world vehicle;

obtaining a simulation route for the virtual vehicle based on the planned route for the real-world vehicle;

causing the virtual vehicle to make the trip by moving along the simulation route in a simulation environment to obtain first simulated vehicle perception data;

modifying the simulation route based on the first simulated vehicle perception data; and

causing the virtual vehicle to finish the trip by moving along the modified simulation route in the simulation environment to obtain second simulated vehicle perception data;

change the planned route based on at least one of the first and second simulated vehicle perception data; and

output trip instructions that are configured to cause the real-world vehicle to move along the planned route which has been changed.

2. The system of claim 1 , wherein:

a user of the real-world vehicle is a first tenant who shares use of the real-world vehicle with one or more other tenants; and

the cloud service provider processor is further caused to transmit a record of simulation to the first tenant.

3. The system of claim 1 , wherein the instructions to cause the virtual vehicle to move along the simulation route in the simulation environment comprise instructions to:

access a first tenant-specific policy for the first tenant; and

cause the virtual vehicle to operate according to the first tenant-specific policy while moving in the simulation environment.

4. The system of claim 3 , further comprising additional programming instructions that are configured to cause the cloud service provider processor to:

access a second tenant-specific policy for a second tenant;

use a high definition map to generate a second planned route for the real-world vehicle;

cause the virtual vehicle to move along the second planned route in the simulation environment according to the second tenant-specific policy; and

output a record of simulation to the second tenant.

5. The system of claim 2 , further comprising additional programming instructions that are configured to cause the cloud service provider processor to, after transmitting the record of simulation to the first tenant and in response to receiving a modification request from the first tenant, using the modification request to further modify the planned route.

6. The system of claim 1 , wherein the instructions to generate the planned route for the real-world vehicle comprise instructions to:

generate a plurality of candidate trajectories for the real-world vehicle from the starting point to the destination in a map; and

select the planned route from the plurality of candidate trajectories.

7. The system of claim 1 , wherein:

the cloud service provider system comprises a data store of a plurality of candidate motion planning systems, each of which is associated with one or more vehicles or one or more fleets; and

the programming instructions are further configured to, before generating the planned route for the real-world vehicle:

identify, from the data store, a candidate motion planning system that is associated with the real-world vehicle or one of the fleets of which the real-world vehicle is a part, and

when selecting the virtual vehicle, selecting the virutal vehicle that is associated with a motion planning system that matches the identified candidate motion planning system.

8. The system of claim 1 , wherein the programming instructions to cause the virtual vehicle to move along the simulation route in the simulation environment comprise instructions to cause a phantom vehicle service to, on a periodic basis, report a current location of the virtual vehicle.

9. The system of claim 1 , wherein the programming instructions to generate the planned route for the real-world vehicle comprise instructions to:

use one or more functions of the associated motion planning system to generate a plurality of candidate trajectories for the real-world vehicle based in part on:

one or more physical capabilities of the real-world vehicle, or

one or more operating restrictions that correspond to a trip service request.

10. A method of generating a planned path for a vehicle, the method comprising:

by a processor that is remote from a real-world vehicle:

receiving a trip request;

generating a planned route for a real-world vehicle in response to the trip request;

simulating a trip by a virtual vehicle by:

selecting the virtual vehicle from a plurality of different virtual vehicles profiled in a datastore by characteristics of real-world vehicles, the virtual vehicle that is selected has an associated motion planning system that corresponds to a motion planning system that is deployed on the real-world vehicle;

obtaining a simulation route for the virtual vehicle based on the planned route for the real-world vehicle;

causing the virtual vehicle to make the trip by moving along the simulation route in a simulation environment to obtain first simulation perception data;

modifying the simulation route based on the first simulated vehicle perception data; and

causing the virtual vehicle to finish the trip by moving along the modified simulation route in the simulation environment to obtain second simulated vehicle perception data;

changing the planned route based on at least one of the first and second simulated vehicle perception data; and

outputting trip instructions that are configured to cause the real-world vehicle to move along the planned route which has been changed.

11. The method of claim 10 , wherein:

a user of the real-world vehicle is a first tenant who shares use of the real-world vehicle with one or more other tenants; and

a record of simulation is transmitted to the first tenant.

12. The method of claim 10 , further comprising:

accessing a second tenant-specific policy for a second tenant;

using a high definition map to generate a second planned route for the real-world vehicle;

causing the virtual vehicle to move along the second planned route in the simulated environment according to the second tenant-specific policy; and

outputting a record of simulation to the second tenant.

13. The method of claim 11 , further comprising, after transmitting the record of simulation to the first tenant and in response to receiving a modification request from the first tenant, using the modification request to further modify the planned route.

14. The method of claim 10 , wherein generating the planned route for the real-world vehicle comprises:

generating a plurality of candidate trajectories for the real-world vehicle from the starting point to the destination in the map; and

selecting the planned route from the plurality of candidate trajectories.

15. The method of claim 10 , further comprising:

accessing a data store of a plurality of candidate motion planning systems, each of which is associated with one or more vehicles or one or more fleets; and

before generating the planned route for the real-world vehicle:

identifying, from the data store, a candidate motion planning system that is associated with the first real-world or one of the fleets of which the real-world vehicle is a part, and

when selecting the virtual vehicle, selecting the virtual vehicle that is associated with a motion planning system that matches the identified candidate motion planning system.

16. The method of claim 10 , wherein causing the virtual vehicle to move along the simulation route in the simulation environment comprises, by a phantom vehicle service on a periodic basis, reporting a current location of the virtual vehicle to the cloud deployment service.

17. The method of claim 10 , wherein the planned route for the real-world vehicle is generated using one or more functions of the associated motion planning system to generate a plurality of candidate trajectories for the real-world vehicle based in part on one or more of the following:

one or more physical capabilities of the real-world vehicle, or

one or more operating restrictions that correspond to a trip service request.

18. A computer program product, comprising:

a memory and programming instructions that are configured to cause a cloud service provider processor that is remote from a real-world vehicle to:

obtain a planned route for the real-world vehicle generated in response to a trip request;

simulate a trip by a virtual vehicle by:

selecting the virtual vehicle from a plurality of different virtual vehicles profiled in a data store by characteristics of real-world vehicles, the virtual vehicle that is selected has an associated motion planning system that corresponds to a motion control system that is deployed on the real-world vehicle;

obtaining a simulation route for the virtual vehicle based on the planned route for the real-world vehicle; and

causing the selected virtual vehicle to move along the simulation route in a simulation environment to obtain first simulated vehicle perception data;

modifying the simulation route based on the first simulated vehicle perception data; and

causing the virtual vehicle to finish the trip by moving along the modified simulated route in the simulation environment to obtain second simulated vehicle perception data;

change the planned route based on at least one of the first and second simulated vehicle perception data; and

output trip instructions that are configured to cause the real-world vehicle to move along the planned route that has been changed.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2024
From: ARGO AI, LLC
To: VOLKSWAGEN GROUP OF AMERICA INVESTMENTS, LLC
Reel/Frame 069177/0099 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 10, 2022
From: VENKATESH, SHUBHASHREE; BRITO, NOE; CHENG, YEE-NING; CHHURA, MADHAV; DOVENOR, SEBASTIAN; DRAKE, JOHN; PAN, JONATHAN; PARRAGA, JASON; PLANT, SCOTT
To: ARGO AI, LLC
Reel/Frame 059221/0958 →
Continuity (3)
Provisional Application 63292140 · Dec 21, 2021
Provisional Application 63252431 · Oct 5, 2021
Related Publication 20230106268A1 · Apr 6, 2023
References Cited (62)
US 6850153B1 · Murakami et al. · 2005 [cited by applicant]
US 9194168B1 · Lu et al. · 2015 [cited by applicant]
US 11228613B2 · Chang et al. · 2022 [cited by applicant]
US 20140108080A1 · Mitchell · 2014 [cited by applicant]
US 20140156327A1 · Cai · 2014 [cited by applicant]
US 20150074013A1 · Schoonmaker · 2015 [cited by examiner]
US 20150142518A1 · Farinha Gomes Felix · 2015 [cited by applicant]
US 20150294403A1 · Chu et al. · 2015 [cited by applicant]
US 20160301698A1 · Katara et al. · 2016 [cited by applicant]
US 20170123421A1 · Kentley et al. · 2017 [cited by applicant]
US 20170339031A1 · Hu et al. · 2017 [cited by applicant]
US 20180003512A1 · Lynch · 2018 [cited by applicant]
US 20180004202A1 · Onaga et al. · 2018 [cited by applicant]
US 20180188042A1 · Chen · 2018 [cited by applicant]
US 20180211541A1 · Rakah et al. · 2018 [cited by applicant]
US 20180216942A1 · Wang · 2018 [cited by applicant]
US 20190009794A1 · Toyoda · 2019 [cited by applicant]
US 20190120640A1 · Ho et al. · 2019 [cited by applicant]
US 20190179336A1 · Colijn et al. · 2019 [cited by applicant]
US 20190228375A1 · Laury et al. · 2019 [cited by applicant]
US 20200116509A1 · Sakaguchi · 2020 [cited by applicant]
US 20200116515A1 · Chadha et al. · 2020 [cited by applicant]
US 20200118075A1 · Yang et al. · 2020 [cited by applicant]
US 20200128101A1 · Meng · 2020 [cited by examiner]
US 20200173808A1 · Beaurepaire et al. · 2020 [cited by applicant]
US 20200191589A1 · Tamai et al. · 2020 [cited by applicant]
US 20200209002A1 · Hou et al. · 2020 [cited by applicant]
US 20200334637A1 · Turner et al. · 2020 [cited by applicant]
US 20200365015A1 · Nguyen · 2020 [cited by applicant]
US 20200372428A1 · Liu et al. · 2020 [cited by applicant]
US 20200380629A1 · Monteil et al. · 2020 [cited by applicant]
US 20200386558A1 · DeLizio · 2020 [cited by applicant]
US 20200396227A1 · Fan et al. · 2020 [cited by applicant]
US 20210033416A1 · Vladimerou · 2021 [cited by applicant]
US 20210090025A1 · Bolton et al. · 2021 [cited by applicant]
US 20210233390A1 · Georgiou · 2021 [cited by applicant]
US 20210241625A1 · Elisha · 2021 [cited by applicant]
US 20210309248A1 · Choe · 2021 [cited by applicant]
US 20220058309A1 · Safira · 2022 [cited by examiner]
US 20220063662A1 · Sprunk · 2022 [cited by applicant]
US 20220412764A1 · Wu · 2022 [cited by applicant]
US 20230015884A1 · Cao · 2023 [cited by applicant]
US 20230085943A1 · Karri · 2023 [cited by examiner]
US 20230086061A1 · Srivastava et al. · 2023 [cited by applicant]
WO 2019153082A1 · 2019 [cited by applicant]
WO 2019213415A1 · 2019 [cited by applicant]
WO 2020228949A1 · 2020 [cited by applicant]
Gonzalez, D. et al., A Review of Motion Planning Techniques for Automated Vehicles, ResearchGate, IEEE Transactions on Intelligent Transportation Systems, vol. 17, No. 4, Apr. 2016. [cited by applicant]
Yurtsever, E. et al., A Survey of Autonomous Driving: Common Practices and Emerging Technologies, IEEE Access, Apr. 2020. [cited by applicant]
Graham Scan Algorithm to find Convex Hull, OpenGenus IQ: Computing Expertise & Legacy, 2022, available at https://iq.opengenus.org/graham-scan-convex-hull/. [cited by applicant]
Amazon Resource Names (ARNs), 2022 Amazon Web Services, Inc., available at https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html. [cited by applicant]
[Deprecated] Embedding Debezium Connectors, Debezium Documentation / Operations / Embedding Debezium, 2022 Debezium Community, available at https://debezium.io/documentation/reference/operations/embedded.html. [cited by applicant]
Chang. M. et al., Argoverse: 3D Tracking and Forecasting with Rich Maps, Nov. 6, 2019. [cited by applicant]
International Search Report and Written Opinion for PCT/US2023/071441 dated Oct. 31, 2023, 9 pages. [cited by applicant]
U.S. Appl. No. 17/650,283, filed Feb. 8, 2022, System and Method for Estimating Arrival Time of a Vehicle at a Destination. [cited by applicant]
U.S. Appl. No. 17/650,286, filed Feb. 8, 2022, System and Method for Generating a Planned Path for a Vehicle Using a Cloud Deployment System. [cited by applicant]
U.S. Appl. No. 17/650,281, filed Feb. 8, 2022, Systems and Methods for Managing Permissions and Authorizing Access to and Use of Services. [cited by applicant]
U.S. Appl. No. 17/650,289, filed Feb. 8, 2022, System and Method for Defining Serviceable Areas. [cited by applicant]
Distributed Multi-AUV Coordination in Naval Mine Countermeasure Missions Sanem Sariel, Tucker Balch and Jason Stack Jan. 30, 2006 (Jan. 30, 2006). [cited by applicant]
On-board Data Mining Steve Tanner, Cara Stein, and Sara J. Graves Jul. 30, 2009 (Jul. 30, 2009). [cited by applicant]
International Search Report and Written Opinion for PCT/US2023/063542 dated May 30, 2023, 13 pages. [cited by applicant]
Karamanis et al., Vehicle redistribution in ride-sourcing markets using convex minimum cost flows, 2020. [cited by applicant]