IP Library Granted Patent US 10,436,885
Granted Patent B2
US 10,436,885 · App. 16/161,035 · Granted Oct 8, 2019

Calibrating sensors mounted on an autonomous vehicle

Inventors: Mark Damon Wheeler (Saratoga, CA); Lin Yang (San Carlos, CA)
Assignee: DEEPMAP INC.
G01S7/497G01C21/3602G01S7/4817G01S7/4972G01S17/023G01S17/42G01S17/89G01S17/936G05D1/0088G05D1/0231G05D1/0248G06T7/13G06T7/33G06T7/55G06T7/80H04N5/04H04N5/232B60R1/00G05D1/0287G05D2201/0213G06K9/00791G06K9/6202G06T2207/10028G06T2207/10048G06T2207/20092G06T2207/20221G06T2207/30241G06T2207/30242G06T2207/30252H04N5/2253H04N5/247
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,436,885
App. No.
16/161,035
Granted
Oct 8, 2019
Kind
B2
Abstract

A system calibrates one or more sensors mounted to an autonomous vehicle. From the one or more sensors, the system identifies a primary sensor and a secondary sensor. The system determines a reference angle for the primary sensor, and based on that reference angle for the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan. The system receives, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time. The system receives, from the secondary sensor, a secondary set of sensor data recorded from the scan-start time to the scan-end time. The system calibrates the primary and secondary sensors by determining a relative transform for transforming points between the first set of scan data and the second set of scan data.

Claims (114)

1. A non-transitory computer readable storage medium having instructions for calibrating light detection and ranging (LIDAR) sensors mounted on an autonomous vehicle encoded thereon that, when executed by a processor, cause the processor to:

identify from a plurality of LIDAR sensors mounted on the autonomous vehicle, a primary LIDAR sensor and a secondary LIDAR sensor;

determine a reference angle for the primary LIDAR sensor;

determine, based on the reference angle of the primary LIDAR sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;

receive, from the primary LIDAR sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;

receive, from the secondary LIDAR sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;

calibrate the primary and secondary LIDAR sensors by determining a relative transform for transforming points between the primary set of scan data and the secondary set of scan data;

generate a high definition map based on scan data recorded by the plurality of LIDAR sensors, the scan data aggregated using the relative transform; and

send signals to the controls of the autonomous vehicle based on the high-definition map.

2. The non-transitory computer readable storage medium of claim 1 , wherein identifying the primary LIDAR sensor comprises:

selecting a LIDAR sensor from the plurality of LIDAR sensors that has a rotational frequency closest to a rate of performing localization for the autonomous vehicle.

3. The non-transitory computer readable storage medium of claim 1 , wherein determining the scan-start time and scan-end time comprises:

detecting that the primary LIDAR sensor rotates past the reference angle a first time; and

determining the scan-start time as the time at which primary LIDAR sensor rotates past the reference angle;

detecting that the primary LIDAR sensor rotates past the reference angle a second time, the second time immediately subsequent to the first time; and

determining the scan-end time as the time at which primary LIDAR sensor rotates past the reference angle a second time.

4. The non-transitory computer readable storage medium of claim 1 , wherein the scan-end time is a scan-start time for a second primary set of scan data recorded by the primary LIDAR sensor.

5. The non-transitory computer readable storage medium of claim 1 , further comprising:

accessing, from a computer memory, a history of scan data recorded by the secondary LIDAR sensor as the primary LIDAR sensor periodically rotates past the reference angle; and

at each scan-start time and scan-end time received from the primary LIDAR sensor, segmenting the history of scan data.

6. The non-transitory computer readable storage medium of claim 5 , wherein segmenting the history of scan data comprises:

identifying a timestamp assigned to portions of scan data recorded by the secondary LIDAR sensor within a threshold of the scan-start time received from the primary LIDAR sensor.

7. The non-transitory computer readable storage medium of claim 1 , wherein the relative transform between the scans of the primary LIDAR sensor and each secondary LIDAR sensor is determined using an iterative closest point technique.

8. A computer-implemented method for calibrating sensors mounted on a vehicle, the method comprising:

identifying from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;

determining a reference angle for the primary sensor;

determining, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;

receiving, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;

receiving, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;

calibrating the primary and secondary sensors by determining a relative transform for transforming points between the primary set of scan data and the secondary set of scan data;

generating a high definition map based on scan data recorded by the plurality of sensors, the scan data aggregated using the relative transform; and

storing the high definition map in a computer readable storage medium.

9. The computer-implemented method of claim 8 , wherein identifying the primary sensor comprises:

selecting a sensor from the plurality of sensors that has a rotational frequency closest to a rate of performing localization for the vehicle.

10. The computer-implemented method of claim 8 , wherein determining the scan-start time and scan-end time comprises:

detecting that the primary sensor rotates past the reference angle a first time; and

determining the scan-start time as the time at which primary sensor rotates past the reference angle;

detecting that the primary sensor rotates past the reference angle a second time, the second time immediately subsequent to the first time; and

determining the scan-end time as the time at which sensor rotates past the reference angle a second time.

11. The computer-implemented method of claim 8 , wherein the scan-end time is a scan-start time for a second primary set of scan data recorded by the primary sensor.

12. The computer-implemented method of claim 8 , further comprising:

accessing, from a computer memory, a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and

at each scan-start time and scan-end time received from the primary sensor, segmenting the history of scan data.

13. The computer-implemented method of claim 5 , wherein segmenting the history of scan data comprises:

identifying a timestamp assigned to portions of scan data recorded by the secondary sensor within a threshold of the scan-start time received from the primary sensor.

14. The computer-implemented method of claim 8 , wherein the relative transform between the scans of the primary sensor and the secondary sensor is determined using an iterative closest point technique.

15. A computer-implemented method for calibrating light detection and ranging (LIDAR) sensors mounted on an autonomous vehicle, the method comprising:

receiving a reference angle for a primary LIDAR sensor;

determining, based on the reference angle of the primary LIDAR sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;

receiving, from the primary LIDAR sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;

receiving, from a secondary LIDAR sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;

calibrating the primary and secondary LIDAR sensors by determining a relative transform for transforming points between the set of scan data and the secondary set of scan data;

generating a high definition map based on scan data recorded by the plurality of LIDAR sensors, the scan data aggregated using the relative transform; and

sending signals to the controls of the autonomous vehicle based on the high-definition map.

16. The computer-implemented method of claim 15 , wherein, determining the scan-start time and scan-end time comprises:

detecting that the primary LIDAR sensor rotates past the reference angle a first time; and

determining the scan-start time as the time at which primary LIDAR sensor rotates past the reference angle;

detecting that the primary LIDAR sensor rotates past the reference angle a second time, the second time immediately subsequent to the first time; and

determining the scan-end time as the time at which primary LIDAR sensor rotates past the reference angle a second time.

17. The computer-implemented method of claim 15 , wherein the scan-end time is a scan-start time for a second primary set of scan data recorded by the primary LIDAR sensor.

18. The computer-implemented method of claim 15 , further comprising:

accessing, from a computer memory, a history of scan data recorded by the secondary LIDAR sensor as the primary LIDAR sensor periodically rotates past the reference angle; and

at each scan-start time and scan-end time received from the primary LIDAR sensor, segmenting the history of scan data.

19. The computer-implemented method of claim 18 , wherein segmenting the history of scan data comprises:

identifying a timestamp assigned to portions of scan data recorded by the secondary LIDAR sensor within a threshold of the scan-start time received from the primary LIDAR sensor.

20. The computer-implemented method of claim 15 , wherein the relative transform between the scans of the primary LIDAR sensor and each secondary LIDAR sensor is determined using an iterative closest point technique.

21. A non-transitory computer readable storage medium storing instructions for calibrating sensors mounted on a vehicle encoded thereon that, when executed by a processor, cause the processor to:

identify from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;

determine a reference angle for the primary sensor;

determine, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;

receive, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;

receive, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;

calibrate the primary and secondary sensors by determining a relative transform for transforming points between the primary set of scan data and the secondary set of scan data;

generating a high definition map based on scan data recorded by the plurality of sensors, the scan data aggregated using the relative transform; and

store the high definition map in a computer readable storage medium.

22. The non-transitory computer readable storage medium of claim 21 , wherein instructions for identifying the primary sensor cause the processor to:

select a sensor from the plurality of sensors that has a rotational frequency closest to a rate of performing localization for the vehicle.

23. The non-transitory computer readable storage medium of claim 21 , wherein instructions for determining the scan-start time and scan-end time cause the processor to:

detect that the primary sensor rotates past the reference angle a first time; and

determine the scan-start time as the time at which primary sensor rotates past the reference angle;

detect that the primary sensor rotates past the reference angle a second time, the second time immediately subsequent to the first time; and

determine the scan-end time as the time at which sensor rotates past the reference angle a second time.

24. The non-transitory computer readable storage medium of claim 21 , wherein the scan-end time is a scan-start time for a second primary set of scan data recorded by the primary sensor.

25. The non-transitory computer readable storage medium of claim 21 , further comprising instructions encoded thereon that, when executed, further cause the processor to:

access, from a computer memory, a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and

at each scan-start time and scan-end time received from the primary sensor, segment the history of scan data.

26. The non-transitory computer readable storage medium of claim 25 , wherein instructions for segmenting the history of scan data cause the processor to:

identify a timestamp assigned to portions of scan data recorded by the secondary sensor within a threshold of the scan-start time received from the primary sensor.

27. The non-transitory computer readable storage medium of claim 21 , wherein the relative transform between the scans of the primary sensor and the secondary sensor is determined using an iterative closest point technique.

28. A computer system for calibrating sensors mounted on a vehicle, the computer system comprising:

a processor; and

a computer-readable storage medium containing computer program code that when executed, causes the processor to:

identify from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;

determine a reference angle for the primary sensor;

determine, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;

receive, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;

receive, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;

calibrate the primary and secondary sensors by determining a relative transform for transforming points between the primary set of scan data and the secondary set of scan data;

generate a high definition map based on scan data recorded by the plurality of sensors, the scan data aggregated using the relative transform; and

store the high definition map in a computer readable storage medium.

29. The computer system of claim 28 , wherein instructions for identifying the primary sensor cause the processor to:

select a sensor from the plurality of sensors that has a rotational frequency closest to a rate of performing localization for the vehicle.

30. The computer system of claim 28 , wherein instructions for determining the scan-start time and scan-end time cause the processor to:

detect that the primary sensor rotates past the reference angle a first time; and

determine the scan-start time as the time at which primary sensor rotates past the reference angle;

detect that the primary sensor rotates past the reference angle a second time, the second time immediately subsequent to the first time; and

determine the scan-end time as the time at which sensor rotates past the reference angle a second time.

31. The computer system of claim 30 , wherein the relative transform between the scans of the primary sensor and the secondary sensor is determined using an iterative closest point technique.

32. The computer system of claim 28 , wherein the scan-end time is a scan-start time for a second primary set of scan data recorded by the primary sensor.

33. The computer system of claim 28 , further comprising instructions encoded thereon that, when executed, further cause the one or more processors to:

access, from a computer memory, a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and

at each scan-start time and scan-end time received from the primary sensor, segment the history of scan data.

34. The computer system of claim 33 , wherein instructions for segmenting the history of scan data cause the processor to:

identify a timestamp assigned to portions of scan data recorded by the secondary sensor within a threshold of the scan-start time received from the primary sensor.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2022
From: DEEPMAP INC.
To: NVIDIA CORPORATION
Reel/Frame 061038/0311 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 16, 2018
From: WHEELER, MARK DAMON; YANG, LIN
To: DEEPMAP INC.
Reel/Frame 047185/0192 →
Continuity (2)
Provisional Application 62574744 · Oct 19, 2017
Related Publication 20190120946A1 · Apr 25, 2019
Cited By (8)
US 12,270,639 US 12,315,197 US 12,384,410 US 12,385,766 US 12,423,866 US 12,450,915 US 12,717,011 US 12,717,012