IP Library Granted Patent US 12,190,541
Granted Patent B2
US 12,190,541 · App. 18/513,247 · Granted Jan 7, 2025

Automated vehicle pose validation

Inventors: Philippe Babin (Pittsburgh, PA); Kunal Anil Desai (San Francisco, CA); Tao V. Fu (Pittsburgh, PA); Gang Pan (Fremont, CA); Xxx Xinjilefu (Pittsburgh, PA)
Assignee: Volkswagen Group of America Investments, LLC
G06T7/74B60W60/001G01S7/4808G01S17/89B60W2420/408G06T2207/10028G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,190,541
App. No.
18/513,247
Granted
Jan 7, 2025
Kind
B2
Abstract

Disclosed herein are system, method, and computer program product embodiments for automated autonomous vehicle pose validation. An embodiment operates by generating a range image from a point cloud solution comprising a pose estimate for an autonomous vehicle. The embodiment queries the range image for predicted ranges and predicted class labels corresponding to lidar beams projected into the range image. The embodiment generates a vector of features from the range image. The embodiment compares a plurality of values to the vector of features using a binary classifier. The embodiment validates the autonomous vehicle pose based on the comparison of the plurality of values to the vector of features using the binary classifier.

Claims (28)

1. A method, comprising:

estimating a pose for an autonomous vehicle based on features extracted from a reference point cloud as compared to semantic features representing surroundings of the autonomous vehicle, wherein the reference point cloud represents data from a plurality of lidar sweeps and is extracted from a high-definition map;

validating the pose estimate as a pose of the autonomous vehicle based on first coordinates derived from a query point cloud generated from a real-time lidar sweep of the surroundings evaluated against second coordinates derived from the reference point cloud by at least one of: aligning the first coordinates with the second coordinates, identifying difference between the first coordinates and the second coordinates, or merging the first coordinates with the second coordinates; and

causing the autonomous vehicle to perform a driving maneuver based on the validated pose estimate.

2. The method of claim 1 , wherein the features extracted from the reference point cloud are based on predicted light pulse distance ranges corresponding to lidar beams from the plurality of lidar sweeps.

3. The method of claim 1 , further comprising invalidating, based on third coordinates derived from another query point cloud generated from another real-time lidar sweep of the environment evaluated against fourth coordinates derived from the reference point cloud, another pose estimate as another pose of the autonomous vehicle.

4. The method of claim 3 , wherein the third coordinates derived from the another query point cloud evaluated against the fourth coordinates derived from the reference point cloud indicates a misalignment of the third coordinates and the fourth coordinates.

5. The method of claim 3 , wherein the another pose estimate is at least one of: received from a global positioning system (GPS), or derived from GPS data.

6. The method of claim 1 , wherein the features extracted from the reference point cloud are compared to the semantic features representing the surroundings of the autonomous vehicle by a binary classifier of the autonomous vehicle.

7. A system, comprising:

a memory; and

at least one processor coupled to the memory and configured to perform operations comprising:

estimating a pose for an autonomous vehicle based on features extracted from a reference point cloud as compared to semantic features representing surroundings of the autonomous vehicle, wherein the reference point cloud represents data from a plurality of lidar sweeps and is extracted from a high-definition map; and

validating the pose estimate as a pose of the autonomous vehicle based on first coordinates derived from a query point cloud generated from a real-time lidar sweep of the surroundings evaluated against second coordinates derived from the reference point cloud by at least one of: aligning the first coordinates with the second coordinates, identifying difference between the first coordinates and the second coordinates, or merging the first coordinates with the second coordinates; and

causing the autonomous vehicle to perform a driving maneuver based on the validated pose estimate.

8. The system of claim 7 , wherein the features extracted from the reference point cloud are based on predicted light pulse distance ranges corresponding to lidar beams from the plurality of lidar sweeps.

9. The system of claim 7 , the operations further comprising invalidating, based on third coordinates derived from another query point cloud generated from another real-time lidar sweep of the environment evaluated against fourth coordinates derived from the reference point cloud, another pose estimate as another pose of the autonomous vehicle.

10. The system of claim 9 , wherein the third coordinates derived from the another query point cloud evaluated against the fourth coordinates derived from the reference point cloud indicates a misalignment of the third coordinates and the fourth coordinates.

11. The system of claim 9 , wherein the another pose estimate is at least one of: received from a global positioning system (GPS), or derived from GPS data.

12. The system of claim 7 , wherein the features extracted from the reference point cloud are compared to the semantic features representing the surroundings of the autonomous vehicle by a binary classifier of the autonomous vehicle.

13. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

estimating a pose for an autonomous vehicle based on features extracted from a reference point cloud as compared to semantic features representing surroundings of the autonomous vehicle, wherein the reference point cloud represents data from a plurality of lidar sweeps and is extracted from a high-definition map; and

validating the pose estimate as a pose of the autonomous vehicle based on first coordinates derived from a query point cloud generated from a real-time lidar sweep of the surroundings evaluated against second coordinates derived from the reference point cloud by at least one of: aligning the first coordinates with the second coordinates, identifying difference between the first coordinates and the second coordinates, or merging the first coordinates with the second coordinates; and

causing the autonomous vehicle to perform a driving maneuver based on the validated pose estimate.

14. The non-transitory computer-readable medium of claim 13 , wherein the features extracted from the reference point cloud are based on predicted light pulse distance ranges corresponding to lidar beams from the plurality of lidar sweeps.

15. The non-transitory computer-readable medium of claim 13 , the operations further comprising invalidating, based on third coordinates derived from another query point cloud generated from another real-time lidar sweep of the environment evaluated against fourth coordinates derived from the reference point cloud, another pose estimate as another pose of the autonomous vehicle.

16. The non-transitory computer-readable medium of claim 15 , wherein the third coordinates derived from the another query point cloud evaluated against the fourth coordinates derived from the reference point cloud indicates a misalignment of the third coordinates and the fourth coordinates.

17. The non-transitory computer-readable medium of claim 15 , wherein the another pose estimate is at least one of: received from a global positioning system (GPS), or derived from GPS data.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 7, 2024
From: ARGO AI, LLC
To: VOLKSWAGEN GROUP OF AMERICA INVESTMENTS, LLC
Reel/Frame 069113/0265 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 4, 2024
From: BABIN, PHILIPPE; DESAI, KUNAL ANIL; FU, TAO V.; PAN, GANG; XINJILEFU, XXX
To: ARGO AI, LLC
Reel/Frame 068805/0356 →
Continuity (2)
Continuation 17541094 · Dec 2, 2021
Related Publication 20240087163A1 · Mar 14, 2024
References Cited (28)
US 9098773B2 · Huang et al. · 2015 [cited by applicant]
US 9489575B1 · Whalen et al. · 2016 [cited by applicant]
US 9514378B2 · Armstrong-Crews et al. · 2016 [cited by applicant]
US 10024664B1 · Gill et al. · 2018 [cited by applicant]
US 10054678B2 · Mei et al. · 2018 [cited by applicant]
US 10220766B2 · Soehner et al. · 2019 [cited by applicant]
US 10416679B2 · Lipson et al. · 2019 [cited by applicant]
US 10445599B1 · Hicks · 2019 [cited by applicant]
US 20170248693A1 · Kim · 2017 [cited by applicant]
US 20180157269A1 · Prasad et al. · 2018 [cited by applicant]
US 20180276912A1 · Zhou · 2018 [cited by applicant]
US 20180284779A1 · Nix · 2018 [cited by applicant]
US 20180299534A1 · LaChapelle et al. · 2018 [cited by applicant]
US 20190129009A1 · Eichenholz et al. · 2019 [cited by applicant]
US 20190339708A1 · Ramirez Llanos · 2019 [cited by examiner]
US 20200159244A1 · Chen et al. · 2020 [cited by applicant]
US 20200301015A1 · Siddiqui et al. · 2020 [cited by applicant]
US 20210278523A1 · Urtasun · 2021 [cited by examiner]
US 20210365712A1 · Lu · 2021 [cited by examiner]
US 20220122324A1 · Jian · 2022 [cited by examiner]
US 20230177719A1 · Babin et al. · 2023 [cited by applicant]
EP 3349146A1 · 2018 [cited by applicant]
EP 3396408A1 · 2018 [cited by applicant]
KR 20110080025A · 2011 [cited by applicant]
WO 2008018906A2 · 2008 [cited by applicant]
WO 2010027795A1 · 2010 [cited by applicant]
International Search Report and Written Opinion of the International Searching Authority directed to related International Patent Application No. PCT/US2022/080734, mailed Dec. 1, 2022; 8 pages. [cited by applicant]
Fazekas, A. et al., “Performance Metrics and Validation Methods for Vehicle Position Estimators”, IEEE Transactions on Intelligent Transportation Systems, Jul. 31, 2020, vol. 21, No. 7. [cited by applicant]
Cited By (1)
US 12,682,488