IP Library Granted Patent US 12,190,607
Granted Patent B2
US 12,190,607 · App. 17/507,182 · Granted Jan 7, 2025

System and method for simultaneous online LIDAR intensity calibration and road marking change detection

Inventor: Khalid Yousif (Milpitas, CA)
G06V20/588G01S7/497G01S17/89G05D1/0274
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,190,607
App. No.
17/507,182
Granted
Jan 7, 2025
Kind
B2
Abstract

A system, method, and computer program for updating calibration lookup tables within an autonomous vehicle or transmitting roadway marking changes between online and offline mapping files is disclosed. A LIDAR sensor may be used for generating an online (rasterized) mapping file with online intensity values which are compared against a correlated offline (rasterized) mapping file having offline intensity values. The online intensity value may be used to acquire a lookup table having a normal distribution that is compared against the offline intensity value. The lookup table may be updated when the offline intensity value is within the normal distribution. Or the vehicle may transmit a roadway marking change when the offline intensity value is outside the normal distribution.

Claims (33)

1. A method for calibrating data used by a sensor system within an autonomous vehicle, comprising:

receiving an online mapping file that includes a plurality of online intensity values indicative of online roadway markings, the online mapping file being generated from a first set of data acquired by a LIDAR sensor coupled to the autonomous vehicle, and wherein the online mapping file is generated based on a path being driven by the autonomous vehicle;

receiving an offline mapping file that includes a plurality of offline intensity values corresponding to offline roadway markings of the path being driven by the autonomous vehicle, wherein the offline mapping file is generated using a second set of data acquired from at least a second LIDAR sensor;

retrieving a lookup table using one of the plurality of online intensity values, wherein the lookup table is a normal distribution having a mean value different from the one of the plurality of online intensity values;

comparing one of the plurality of offline intensity values with the mean value of the normal distribution to determine whether the one of the plurality of offline intensity values is within the normal distribution; and

updating the normal distribution for the lookup table when the one of the plurality of offline intensity values is within the normal distribution.

2. The method of claim 1 , wherein the online roadway marking is detected when the one of the plurality of online intensity values exceeds a predetermined threshold.

3. The method of claim 1 , further comprising:

controlling operation of the autonomous vehicle based on the online roadway markings.

4. The method of claim 1 , wherein the one of the plurality of offline intensity values representative of the offline roadway marking is different than the one of the plurality of online intensity values representative of the online roadway marking.

5. The method of claim 4 , further comprising:

publishing, to a remote device, the online roadway marking is different than the offline roadway marking when the one of the plurality of offline intensity values is outside the normal distribution.

6. The method of claim 1 , wherein the online mapping file is a rasterized data file.

7. The method of claim 6 , wherein the rasterized data file is a gray-scale data file.

8. The method of claim 1 , wherein the offline mapping file is received from remote computing device.

9. A system for calibrating data used by a sensor system within an autonomous vehicle, comprising:

a LIDAR sensor configured coupled to the autonomous vehicle, the LIDAR sensor configured to acquire a first set of data used to generate an online mapping file that includes a plurality of online intensity values indicative of online roadway markings, wherein the online mapping file is generated based on a path being driven by the autonomous vehicle;

a controller configured to:

receive the online mapping file;

receive an offline mapping file that includes a plurality of offline intensity values corresponding to offline roadway marking of the path being driven by the autonomous vehicle, wherein the offline mapping file is generated using a second set of data acquired from at least a second LIDAR sensor;

retrieve at least one lookup table using one of the plurality of online intensity values, wherein the one lookup table is a normal distribution having a mean value different from the one of the plurality of online intensity values; and

update the normal distribution for the at least one lookup table when at least one of the plurality of offline intensity values is within the normal distribution.

10. The system of claim 9 , wherein the online roadway marking is detected when at least one of the plurality of online intensity values exceeds a predetermined threshold.

11. The system of claim 9 , wherein the controller is further configured to control operation of the autonomous vehicle based on the online roadway markings.

12. The system of claim 9 , wherein the one of the plurality of offline intensity values representative of the offline roadway marking is different than the at least one of the plurality of online intensity values representative of the online roadway marking.

13. The system of claim 12 , wherein the controller is further configured to publish, to a remote device, the online roadway marking is different than the offline roadway marking when the at least one of the plurality of offline intensity values is outside the normal distribution.

14. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

receiving an online mapping file that includes a plurality of online intensity values indicative of online roadway markings, the online mapping file being generated from a first set of data acquired by a LIDAR sensor coupled to an autonomous vehicle, and wherein the online mapping file is generated based on a path being driven by the autonomous vehicle;

receiving an offline mapping file that includes a plurality of offline intensity values corresponding to offline roadway markings of the path being driven by the autonomous vehicle, wherein the offline mapping file is generated using a second set of data acquired from at least a second LIDAR sensor;

retrieving one lookup table using one of the plurality of online intensity values, wherein the one lookup table is a normal distribution having a mean value different from the one of the plurality of online intensity values; and

updating the normal distribution for the lookup table when the one of the plurality of offline intensity values is within the normal distribution.

15. The non-transitory computer-readable medium of claim 14 , further comprising: detecting an online roadway marking along the path being driven by the autonomous vehicle using the online mapping file, wherein the online roadway marking is detected when the one of the plurality of online intensity values exceeds a predetermined threshold.

16. The non-transitory computer-readable medium of claim 15 , further comprising: publishing, to a remote device, the online roadway marking is different than the offline roadway marking when the one of the at least one of the plurality of offline intensity values is outside the normal distribution, and wherein the one of the plurality of offline intensity values representative of the offline roadway marking is different than the one of the plurality of online intensity values representative of the online roadway marking.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2023
From: ARGO AI, LLC
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 062936/0548 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2023
From: ARGO AI, LLC
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 063025/0346 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 21, 2021
From: YOUSIF, KHALID
To: ARGO AI, LLC
Reel/Frame 057868/0062 →
Continuity (1)
Related Publication 20230126833A1 · Apr 27, 2023
References Cited (17)
US 9395192B1 · Silver · 2016 [cited by examiner]
US 9530062B2 · Nguyen et al. · 2016 [cited by applicant]
US 10657390B2 · Mei et al. · 2020 [cited by applicant]
US 20100100268A1 · Zhang · 2010 [cited by examiner]
US 20160209846A1 · Eustice · 2016 [cited by examiner]
US 20190391268A1 · Houts · 2019 [cited by examiner]
US 20200160559A1 · Urtasun · 2020 [cited by examiner]
US 20200175720A1 · Hsu · 2020 [cited by examiner]
US 20200226790A1 · Alvarez · 2020 [cited by examiner]
US 20200340817A1 · Schomerus · 2020 [cited by examiner]
US 20210063199A1 · Akbarzadeh et al. · 2021 [cited by applicant]
US 20210097308A1 · Kumar et al. · 2021 [cited by applicant]
US 20210172756A1 · Wheeler et al. · 2021 [cited by applicant]
US 20230126833A1 · Yousif · 2023 [cited by examiner]
KR 20210089172A · 2021 [cited by applicant]
Extended European Search Report of European application No. 22202559.5 dated Mar. 7, 2023, 10 pages. [cited by applicant]
Jeong et al., “Lidar Intensity Calibration for Road Marking Extraction”, 15th International Conference on Ubiquitous Robots (UR), 2018, 6 pages. [cited by applicant]
Cited By (1)
US 12,266,187