IP Library Granted Patent US 12,233,910
Granted Patent B2
US 12,233,910 · App. 17/181,455 · Granted Feb 25, 2025

Indoor localization of autonomous vehicles

Inventors: Benjamin David Ballard (Sewickley, PA); Matthew Puchalski (Pittsburgh, PA); Dale Humbert Lord (Gibsonia, PA); Jason Paul Ziglar (Pittsburgh, PA); John Martin Seminatore (Pittsburgh, PA); Adela Hsien-Neng Wee (Glenshaw, PA); Kevin Scott Swanson (Detroit, MI)
Assignee: Volkswagen Group of America Investments, LLC
B60W60/0025G01C21/206H04W4/029B60W2420/403B60W2420/408
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,233,910
App. No.
17/181,455
Granted
Feb 25, 2025
Kind
B2
Abstract

Devices, systems, and methods are provided for indoor localization. A self-driving vehicle may detect a first fiducial marker located at a first location within a building, wherein the first fiducial marker comprises first fiducial marker information associated with the first location. The self-driving vehicle may retrieve the first fiducial marker information from the first fiducial marker. The self-driving vehicle may generate localization information of the self-driving vehicle based on the first fiducial marker information. The self-driving vehicle may utilize the localization information to transition to a second location within the building, wherein the second location comprises a second fiducial marker.

Claims (48)

1. A self-driving vehicle, the self-driving vehicle comprising processing circuitry coupled to storage and a sensor system, the processing circuitry configured to:

detect, using a camera of the sensor system, a transitional fiducial marker located at a transition point corresponding to a building;

transition the processing circuitry from utilizing global positioning system (GPS) signals to utilizing one or more fiducial markers placed within the building for in-building navigation, using information retrieved from the transitional fiducial marker;

detect, using the camera of the sensor system, a first fiducial marker of the one more fiducial markers, wherein the first fiducial marker is located at a first location within the building;

retrieve, from the first fiducial marker, first fiducial marker information associated with the first location and transform information associated with a second location corresponding to a second fiducial marker of the one more fiducial markers, wherein the transform information is used by the self-driving vehicle to determine its pose relative to the second location;

generate localization information of the self-driving vehicle based on the first fiducial marker information; and

transition the self-driving vehicle to the second location within the building based on the localization information and the transform information.

2. The self-driving vehicle of claim 1 , wherein the transform information includes transformation matrices that describe rotation and translation of the self-driving vehicle with respect to the second location.

3. The self-driving vehicle of claim 1 , wherein the first fiducial marker is located in the line of sight of the self-driving vehicle.

4. The self-driving vehicle of claim 1 , wherein the second fiducial marker is located in the line of sight of the self-driving vehicle.

5. The self-driving vehicle of claim 1 , wherein the first fiducial marker information comprises global positioning system (GPS) information of the first location, a floor number of the building, coordinates of the fiducial marker, elevation of the fiducial marker, or a map location of the fiducial marker.

6. The self-driving vehicle of claim 1 , wherein retrieving the first fiducial marker information comprises the processing circuitry further configured to:

capture an image of the first fiducial marker;

analyze the image of the first fiducial marker; and

decode the first fiducial marker information of the first fiducial marker.

7. The self-driving vehicle of claim 1 , wherein the processing circuitry is further configured to calculate a distance from the self-driving vehicle to the first fiducial marker.

8. The self-driving vehicle of claim 1 , wherein the processing circuitry is further configured to:

detect that the second fiducial marker is moving away from the line of sight of the self-driving vehicle as the self-driving vehicle transitions from the second location to a third location;

detect a third fiducial marker of the one more fiducial markers located at the second location of the building, wherein the third fiducial marker is in the line of sight of the self-driving vehicle, and wherein the third fiducial marker comprises third fiducial marker information associated with the second location.

9. A method comprising:

detecting, using a camera of a sensor system of a self-driving vehicle, a transitional fiducial marker located at a transition point corresponding to a building;

transitioning a processing circuitry of the self-driving vehicle from utilizing global positioning system (GPS) signals to utilizing one or more fiducial markers placed within the building for in-building navigation, using information retrieved from the transitional fiducial marker;

detecting, using the camera of the sensor system of the self-driving vehicle, a first fiducial marker of the one more fiducial markers, wherein the first fiducial marker is located at a first location within the building;

retrieving, from the first fiducial marker, first fiducial marker information associated with the first location and transform information associated with a second location corresponding to a second fiducial marker of the one more fiducial markers, wherein the transform information is used by the self-driving vehicle to determine its pose relative to the second location;

generating localization information of the self-driving vehicle based on the first fiducial marker information; and

transition the self-driving vehicle to the second location within the building based on the localization information and the transform information.

10. The method of claim 9 , wherein the transform information includes transformation matrices that describe rotation and translation of the self-driving vehicle with respect to the second location.

11. The method of claim 9 , wherein the first fiducial marker is located in the line of sight of the self-driving vehicle.

12. The method of claim 9 , wherein the second fiducial marker is located in the line of sight of the self-driving vehicle.

13. The method of claim 9 , wherein the first fiducial marker information comprises global positioning system (GPS) information of the first location, a floor number of the building, coordinates of the fiducial marker, elevation of the fiducial marker, or a map location of the fiducial marker.

14. The method of claim 9 , wherein retrieving the first fiducial marker information comprises:

capturing an image of the first fiducial marker;

analyzing the image of the first fiducial marker; and

decoding the first fiducial marker information of the first fiducial marker.

15. The method of claim 9 , further comprising calculate a distance from the self-driving vehicle to the first fiducial marker.

16. The method of claim 9 , further comprising:

detecting that the second fiducial marker is moving away from the line of sight of the self-driving vehicle as the self-driving vehicle transitions from the second location to a third location;

detecting a third fiducial marker of the one more fiducial markers located at the second location of the building, wherein the third fiducial marker is in the line of sight of the self-driving vehicle, and wherein the third fiducial marker comprises third fiducial marker information associated with the second location.

17. A non-transitory computer-readable medium storing computer-executable instructions which when executed by one or more processors of a self-driving vehicle result in performing operations comprising:

detecting, using a camera of a sensor system of the self-driving vehicle, a transitional fiducial marker located at a transition point corresponding to a building;

transitioning a processing circuitry of the self-driving vehicle from utilizing global positioning system (GPS) signals to utilizing one or more fiducial markers placed within the building for in-building navigation, using information retrieved from the transitional fiducial marker;

detecting, using the camera of the sensor system of the self-driving vehicle, a first fiducial marker of the one more fiducial markers, wherein the first fiducial marker is located at a first location within the building;

retrieving, from the first fiducial marker, first fiducial marker information associated with the first location and transform information associated with a second location corresponding to a second fiducial marker of the one more fiducial markers, wherein the transform information is used by the self-driving vehicle to determine its pose relative to the second location;

generating localization information of the self-driving vehicle based on the first fiducial marker information; and

transition the self-driving vehicle to the second location within the building based on the localization information and the transform information.

18. The non-transitory computer-readable medium of claim 17 , wherein the transform information includes transformation matrices that describe rotation and translation of the self-driving vehicle with respect to the second location.

19. The non-transitory computer-readable medium of claim 17 , wherein the first fiducial marker is located in the line of sight of the self-driving vehicle.

20. The non-transitory computer-readable medium of claim 17 , wherein the second fiducial marker is located in the line of sight of the self-driving vehicle.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 7, 2024
From: ARGO AI, LLC
To: VOLKSWAGEN GROUP OF AMERICA INVESTMENTS, LLC
Reel/Frame 069113/0265 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2021
From: BALLARD, BENJAMIN DAVID; PUCHALSKI, MATTHEW; LORD, DALE HUMBERT; ZIGLAR, JASON PAUL; SEMINATORE, JOHN MARTIN; WEE, ADELA HSIEN-NENG; SWANSON, KEVIN SCOTT
To: ARGO AI, LLC
Reel/Frame 055385/0329 →
Continuity (1)
Related Publication 20220266867A1 · Aug 25, 2022
References Cited (28)
US 10471992B2 · Brennan et al. · 2019 [cited by applicant]
US 20160033963A1 · Noh · 2016 [cited by applicant]
US 20160178382A1 · Kojo et al. · 2016 [cited by applicant]
US 20170061186A1 · Laurent · 2017 [cited by examiner]
US 20170297625A1 · Yion et al. · 2017 [cited by applicant]
US 20170313307A1 · Nordbruch · 2017 [cited by applicant]
US 20170323129A1 · Davidson · 2017 [cited by examiner]
US 20180004219A1 · Aldred et al. · 2018 [cited by applicant]
US 20180111274A1 · Seok et al. · 2018 [cited by applicant]
US 20180301031A1 · Naamani et al. · 2018 [cited by applicant]
US 20180306587A1 · Holz · 2018 [cited by applicant]
US 20180330172A1 · Lee · 2018 [cited by examiner]
US 20190016384A1 · Carlson et al. · 2019 [cited by applicant]
US 20190056483A1 · Bradley et al. · 2019 [cited by applicant]
US 20190080612A1 · Weissman et al. · 2019 [cited by applicant]
US 20190265722A1 · Haeusler et al. · 2019 [cited by applicant]
US 20190332096A1 · Porter · 2019 [cited by examiner]
US 20200050894A1 · Park · 2020 [cited by applicant]
US 20200089237A1 · Whitaker · 2020 [cited by examiner]
US 20200164510A1 · Gong et al. · 2020 [cited by applicant]
US 20200301445A1 · Jourdan · 2020 [cited by examiner]
KR 1020160015987A · 2016 [cited by applicant]
WO 2019055281A2 · 2019 [cited by applicant]
WO 2020177492A1 · 2020 [cited by applicant]
Huang et al. “Vision-based Semantic Mapping and Localization for Autonomous Indoor Parking”, 2018 IEEE Intelligent Vehicles Symposium, Sep. 26, 2018, pp. 636-641, vol. 4, IEEE, New York, NY. [cited by applicant]
Fang et al. “Marker-Based Mapping and Localization for Autonomous Valet Parking”, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 1-6, IEEE, New York, NY. [cited by applicant]
Choi et al. “In-Lane Localization and Ego-Lane Identification Method Based on Highway Lane Endpoints”, Hindawi Journal of Advanced Transportation, Feb. 1, 2020, pp. 1-16, vol. 2020, Cairo, Egypt. [cited by applicant]
International Search Report and Written Opinion of the International Searching Authority directed to related International Patent Application No. PCT/US2022/016666, mailed May 31, 2022; 10 pages. [cited by applicant]