IP Library Granted Patent US 11,375,119
Granted Patent B2
US 11,375,119 · App. 16/538,802 · Granted Jun 28, 2022

Calibrating sensors mounted on an autonomous vehicle

Inventors: Mark Damon Wheeler (Saratoga, CA); Lin Yang (San Carlos, CA)
Assignee: NVIDIA CORPORATION
H04N5/2329G01C21/3602G01S7/4817G01S7/497G01S7/4972G01S17/42G01S17/86G01S17/87G01S17/89G01S17/931G05D1/0088G05D1/0231G05D1/0248G06T7/13G06T7/33G06T7/55G06T7/80H04N5/04H04N5/232H04N13/106B60R1/00G05D1/0287G05D2201/0213G06T2207/10028G06T2207/10048G06T2207/20092G06T2207/20221G06T2207/30241G06T2207/30242G06T2207/30252G06V10/751G06V20/56H04N5/2253H04N5/247
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,375,119
App. No.
16/538,802
Granted
Jun 28, 2022
Kind
B2
Abstract

A system calibrates one or more sensors mounted to an autonomous vehicle. From the one or more sensors, the system identifies a primary sensor and a secondary sensor. The system determines a reference angle for the primary sensor, and based on that reference angle for the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan. The system receives, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time. The system receives, from the secondary sensor, a secondary set of sensor data recorded from the scan-start time to the scan-end time. The system calibrates the primary and secondary sensors by determining a relative transform for transforming points between the first set of scan data and the second set of scan data.

Claims (63)

1. A non-transitory computer readable storage medium having instructions for calibrating sensors mounted on a vehicle encoded thereon that, when executed by a processor, cause the processor to:

identify, from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;

obtain a reference angle for the primary sensor;

determine, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;

receive, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;

receive, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;

align the primary set of scan data with the secondary set of scan data based on one or more of the scan-start time and the scan-end time; and

send signals to controls of the vehicle based on the alignment of the primary set of scan data and the secondary set of scan data.

2. The non-transitory computer readable storage medium of claim 1 , further comprising instructions that cause the processor to:

obtain a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and

at each scan-start time and scan-end time received from the primary sensor, segment the history of scan data.

3. The non-transitory computer readable storage medium of claim 1 , wherein the instructions for aligning the primary set of scan data and the secondary set of scan data further cause the processor to:

identify a timestamp assigned to a portion of the secondary set of scan data associated with the reference angle; and

align the primary set of scan data with the secondary set of scan data based on the timestamp.

4. The non-transitory computer readable storage medium of claim 1 , wherein the scan-end time representing the end of a first rotation by the primary sensor is a scan-start time for a second rotation by the secondary sensor.

5. The non-transitory computer readable storage medium of claim 1 , wherein the primary sensor and the secondary sensor are light detection and ranging (LIDAR) sensors.

6. The non-transitory computer readable storage medium of claim 1 , wherein the instructions for aligning the primary set of scan data and the secondary set of scan data further cause the processor to:

identify one or more shared features recorded in both a segment of the primary set of scan data and a segment of the secondary set of scan data; and

align the segment of the primary set of scan data with the segment of the secondary set of scan data by overlapping the shared features.

7. The non-transitory computer readable storage medium of claim 1 , further comprising instructions that cause the processor to:

obtain an initial orientation of the secondary sensor, wherein the aligning the primary set of scan data with the secondary set of scan data is further based on the initial orientation.

8. A computer-implemented method for calibrating sensors mounted on a vehicle, the method comprising:

identifying, from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;

obtaining a reference angle for the primary sensor;

determining, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;

receiving, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;

receiving, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;

aligning the primary set of scan data with the secondary set of scan data based on one or more of the scan-start time and the scan-end time; and

sending signals to controls of the vehicle based on the alignment of the primary set of scan data and the secondary set of scan data.

9. The computer-implemented method of claim 8 , further comprising:

obtaining a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and

at each scan-start time and scan-end time received from the primary sensor, segmenting the history of scan data.

10. The computer-implemented method of claim 8 , wherein aligning the primary set of scan data and the secondary set of scan data further comprises:

identifying a timestamp assigned to a portion of the secondary set of scan data associated with the reference angle; and

align the primary set of scan data with the secondary set of scan data based on the timestamp.

11. The computer-implemented method of claim 8 , wherein the scan-end time, representing the end of a first rotation by the primary sensor, is a scan-start time for a second rotation by the secondary sensor.

12. The computer-implemented method of claim 8 , wherein the primary sensor and the secondary sensor are light detection and ranging (LIDAR) sensors.

13. The computer-implemented method of claim 8 , wherein aligning the primary set of scan data and the secondary set of scan data further comprises:

identifying one or more shared features recorded in both a segment of the primary set of scan data and a segment of the secondary set of scan data; and

aligning the segment of the primary set of scan data with the segment of the secondary set of scan data by overlapping the shared features.

14. The computer-implemented method of claim 8 , further comprising:

obtaining an initial orientation of the secondary sensor, wherein the aligning the primary set of scan data with the secondary set of scan data is further based on the initial orientation.

15. A computer system for calibrating sensors mounted on a vehicle, the computer system comprising:

a processor; and

a computer readable storage medium having instructions for calibrating sensors mounted on an vehicle encoded thereon that, when executed by the processor, cause the processor to:

identify, from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;

obtain a reference angle for the primary sensor;

determine, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;

receive, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;

receive, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;

align the primary set of scan data with the secondary set of scan data based on one or more of the scan-start time, and the scan-end time; and

send signals to controls of the vehicle based on the alignment of the primary set of scan data and the secondary set of scan data.

16. The computer system of claim 15 , further comprising instructions that cause the processor to:

obtain a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and

at each scan-start time and scan-end time received from the primary sensor, segment the history of scan data.

17. The computer system of claim 15 , wherein the instructions for aligning the primary set of scan data and the secondary set of scan data further cause the processor to:

identify a timestamp assigned to a portion of the secondary set of scan data recorded associated with the reference angle; and

align the primary set of scan data with the secondary set of scan data based on the timestamp.

18. The computer system of claim 15 , wherein the scan-end time representing the end of a first rotation by the primary sensor is a scan-start time for a second rotation by the secondary sensor.

19. The computer system of claim 15 , wherein the primary sensor and the secondary sensor are light detection and ranging (LIDAR) sensors.

20. The computer system of claim 15 , wherein the instructions for aligning the primary set of scan data and the secondary set of scan data further cause the processor to:

identify one or more shared features recorded in both a segment of the primary set of scan data and a segment of the secondary set of scan data; and

align the segment of the primary set of scan data with the segment of the secondary set of scan data by overlapping the shared features.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2022
From: DEEPMAP INC.
To: NVIDIA CORPORATION
Reel/Frame 061038/0311 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 13, 2019
From: WHEELER, MARK DAMON; YANG, LIN
To: DEEPMAP INC.
Reel/Frame 050043/0346 →
Continuity (3)
Continuation 16161035 · Oct 15, 2018
Provisional Application 62574744 · Oct 19, 2017
Related Publication 20200120253A1 · Apr 16, 2020