IP Library Granted Patent US 11,760,386
Granted Patent B2
US 11,760,386 · App. 17/066,108 · Granted Sep 19, 2023

Systems and methods for vehicle-to-vehicle communications for improved autonomous vehicle operations

Inventors: Sivabalan Manivasagam (Toronto, CA); Ming Liang (Toronto, CA); Bin Yang (Toronto, CA); Wenyuan Zeng (Toronto, CA); Raquel Urtasun (Toronto, CA); Tsun-Hsuan Wang (Cambridge, CA)
Assignee: UATC, LLC
B60W60/0027G06N3/044G06N3/08G08G1/0104G08G1/0112G08G1/22H04W4/38H04W4/46B60W2556/65
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,760,386
App. No.
17/066,108
Granted
Sep 19, 2023
Kind
B2
Abstract

Systems and methods for vehicle-to-vehicle communications are provided. An example computer-implemented method includes obtaining from a first autonomous vehicle, by a computing system onboard a second autonomous vehicle, a first compressed intermediate environmental representation. The first compressed intermediate environmental representation is indicative of at least a portion of an environment of the second autonomous vehicle and is based at least in part on sensor data acquired by the first autonomous vehicle at a first time. The method includes generating, by the computing system, a first decompressed intermediate environmental representation by decompressing the first compressed intermediate environmental representation. The method includes determining, by the computing system, a first time-corrected intermediate environmental representation based at least in part on the first decompressed intermediate environmental representation. The first time-corrected intermediate environmental representation corresponds to a second time associated with the second autonomous vehicle.

Claims (49)

1. A computer-implemented method for vehicle-to-vehicle communications, the method comprising:

obtaining from a first autonomous vehicle, by a computing system comprising one or more computing devices onboard a second autonomous vehicle, a first compressed intermediate environmental representation,

wherein the first compressed intermediate environmental representation is indicative of at least a portion of an environment of the second autonomous vehicle and is based at least in part on sensor data acquired by the first autonomous vehicle at a first time;

generating, by the computing system, a first decompressed intermediate environmental representation by decompressing the first compressed intermediate environmental representation; and

determining, by the computing system, a first time-corrected intermediate environmental representation based at least in part on the first decompressed intermediate environmental representation, wherein the first time-corrected intermediate environmental representation is indicative of a time difference between the first time and a second time at which the second autonomous vehicle obtains sensor data of the environment to account for the time difference.

2. The computer-implemented method of claim 1 , wherein determining the first time-corrected intermediate environmental representation comprises:

generating, by the computing system, the first time-corrected intermediate environmental representation based at least in part on a machine-learned time correction model,

wherein the machine-learned time correction model is configured to adjust the first decompressed intermediate environmental representation to account for the time difference between the first time and the second time.

3. The computer-implemented method of claim 2 , wherein the machine-learned time correction model comprises a neural network.

4. The computer-implemented method of claim 1 , wherein the first time is associated with a sensor timestamp of the first autonomous vehicle.

5. The computer-implemented method of claim 1 , wherein the second time is indicative of a time at which the second autonomous vehicle obtains the sensor data through one or more sensors of the second autonomous vehicle.

6. The computer-implemented method of claim 1 , further comprising:

determining, by the computing system, an updated intermediate environmental representation based at least in part on the first time-corrected intermediate environmental representation.

7. The computer-implemented method of claim 6 , further comprising:

determining, by the computing system, an autonomy output based at least in part on the updated intermediate environmental representation.

8. The computer-implemented method of claim 7 , further comprising:

generating, by the computing system, a motion plan for the second autonomous vehicle based at least in part on the autonomy output.

9. The computer-implemented method of claim 8 , further comprising:

initiating, by the computing system, a motion control of the second autonomous vehicle based at least in part on the motion plan.

10. A computing system comprising:

a machine-learned time correction model configured to compensate for time differences between a plurality of times;

one or more processors; and

one or more tangible, non-transitory, computer readable media that collectively store instructions that when executed by the one or more processors cause the computing system to perform operations, the operations comprising:

obtaining a first compressed intermediate environmental representation from a first autonomous vehicle, wherein the first compressed intermediate environmental representation is based at least in part on sensor data acquired by the first autonomous vehicle at a first time;

generating a first decompressed intermediate environmental representation by decompressing the first compressed intermediate environmental representation;

determining, using the machine-learned time correction model, a first time-corrected intermediate environmental representation indicative of a time difference between the first time and a second time based at least in part on the first decompressed intermediate environmental representation, wherein the first time-corrected intermediate environmental representation is adjusted based at least in part on the time difference associated with a second autonomous vehicle; and

generating an updated intermediate environmental representation based at least in part on the first time-corrected intermediate environmental representation.

11. The computing system of claim 10 , wherein the first time is associated with a sensor timestamp of the first autonomous vehicle.

12. The computing system of claim 10 , wherein the first time and the second time are based on global positioning system data.

13. The computing system of claim 10 , wherein generating the updated intermediate environmental representation based at least in part on the first time-corrected intermediate environmental representation comprises:

aggregating, using a machine-learned aggregation model, the first time-corrected intermediate environmental representation and a second intermediate environmental representation generated by the second autonomous vehicle.

14. The computing system of claim 13 , wherein the machine-learned aggregation model is a graph neural network comprising a plurality of nodes.

15. The computing system of claim 14 , wherein the machine-learned aggregation model is configured to initialize a node state of at least one node based at least in part on the time difference between the first time and the second time.

16. The computing system of claim 10 , wherein the operations further comprise:

generating a motion plan based at least in part on the updated intermediate environmental representation; and

initiating a motion control of the second autonomous vehicle based at least in part on the motion plan.

17. An autonomous vehicle comprising:

one or more processors; and

one or more tangible, non-transitory, computer readable media that collectively store instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising:

obtaining a first compressed intermediate environmental representation from another autonomous vehicle, wherein the first compressed intermediate environmental representation is based at least in part on sensor data acquired by the other autonomous vehicle at a first time;

generating a first decompressed intermediate environmental representation by decompressing the first compressed intermediate environmental representation;

determining a first time-corrected intermediate environmental representation based at least in part on the first decompressed intermediate environmental representation and one or more machine-learned models, wherein the first time-corrected intermediate environmental representation is indicative of a time difference between the first time and a second time and, wherein the first time-corrected intermediate environmental representation is adjusted based at least in part on the time difference associated with the autonomous vehicle; and

performing one or more autonomy operations of the autonomous vehicle based at least in part on the first time-corrected intermediate environmental representation.

18. The autonomous vehicle of claim 17 , wherein performing the one or more autonomy operations of the autonomous vehicle based at least in part on the first time-corrected intermediate environmental representation comprises:

generating an autonomy output based at least in part on the first time-corrected intermediate environmental representation.

19. The autonomous vehicle of claim 18 , wherein generating the autonomy output based at least in part on the first time-corrected intermediate environmental representation comprises:

generating an updated intermediate environmental representation based at least in part on the first time-corrected intermediate environmental representation; and

generating the autonomy output based at least in part on the updated intermediate environmental representation.

20. The autonomous vehicle of claim 18 , wherein the autonomy output is indicative of perception data and prediction data associated with the autonomous vehicle.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2024
From: UATC, LLC
To: AURORA OPERATIONS, INC.
Reel/Frame 067733/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2022
From: UBER TECHNOLOGIES, INC.
To: UATC, LLC
Reel/Frame 058962/0140 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 14, 2022
From: YANG, BIN; MANIVASAGAM, SIVABALAN; WANG, TSU-SHUAN; ZENG, WENYUAN
To: UATC, LLC
Reel/Frame 058655/0582 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2021
From: URTASUN SOTIL, RAQUEL
To: UBER TECHNOLOGIES, INC.
Reel/Frame 056969/0695 →
EMPLOYMENT AGREEMENT Recorded Mar 24, 2021
From: LIANG, MING
To: UBER TECHNOLOGIES, INC.
Reel/Frame 056942/0902 →