IP Library Granted Patent US 11,482,008
Granted Patent B2
US 11,482,008 · App. 16/920,105 · Granted Oct 25, 2022

Directing board repositioning during sensor calibration for autonomous vehicles

Inventors: Ziqiang Huang (Palo Alto, CA); Lin Yang (San Carlos, CA); Mark Damon Wheeler (Saratoga, CA)
Assignee: NVIDIA CORPORATION
G06V20/56B60W60/00G01S7/4814G01S7/4817G01S7/497G01S17/10G01S17/89G01S17/894G01S17/931G06T7/74G06V10/751B60W2420/42B60W2420/52G05D1/0231
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,482,008
App. No.
16/920,105
Granted
Oct 25, 2022
Kind
B2
Abstract

According to an aspect of an embodiment, operations may comprise determining a target position and orientation for a calibration board with respect to a camera of a vehicle, detecting a first position and orientation of the calibration board with respect to the camera of the vehicle, determining instructions for moving the calibration board from the first position and orientation to the target position and orientation, transmitting the instructions to a device, detecting a second position and orientation of the calibration board, determining whether the second position and orientation is within a threshold of matching the target position and orientation, and, in response to determining that the second position and orientation is within the threshold of matching the target position and orientation, capturing one or more calibration camera images using the camera and calibrating one or more sensors of the vehicle using the one or more calibration camera images.

Claims (83)

1. A computer-implemented method, comprising:

determining a target position and orientation for a calibration board with respect to a camera of a vehicle;

detecting a first position and orientation of the calibration board with respect to the camera of the vehicle;

determining instructions for moving the calibration board from the first position and orientation to the target position and orientation;

transmitting the instructions to a device proximate the calibration board;

detecting a second position and orientation of the calibration board with respect to the camera of the vehicle;

determining whether the second position and orientation is within a threshold of matching the target position and orientation; and

in response to determining that the second position and orientation is within the threshold of matching the target position and orientation, capturing one or more calibration camera images using the camera and calibrating one or more sensors of the vehicle using the one or more calibration camera images.

2. The computer-implemented method of claim 1 , wherein:

in response to receiving the instructions, the device is configured to control a machine to automatically reposition the calibration board.

3. The computer-implemented method of claim 1 , wherein:

the device comprises a mobile device; and

in response to receiving the instructions, the mobile device is configured to present corresponding directions to a user to guide the user in manually repositioning the calibration board.

4. The computer-implemented method of claim 3 , further comprising presenting an indication on the mobile device that the second position and orientation of the calibration board is within the threshold of matching the target position and orientation.

5. The computer-implemented method of claim 1 , wherein the capturing of the one or more calibration camera images using the camera comprises:

(a) capturing a preliminary camera image using the camera;

(b) detecting a region of the preliminary camera image covering a pattern of the calibration board;

(c) determining an average value of pixels within the region of the preliminary camera image;

(d) until the average value of the pixels is within an exposure threshold of a predetermined value, sending a signal to the camera to adjust an exposure of the camera to bring the average value of the pixels closer to the predetermined value and repeating (a)-(c); and

(e) capturing the one or more calibration camera images using the camera.

6. The computer-implemented method of claim 5 , wherein the adjusting of the exposure of the camera comprises adjusting a size of an aperture of the camera, adjusting a shutter speed of the camera, or adjusting a sensitivity of a sensor of the camera toward light, or some combination thereof.

7. The computer-implemented method of claim 1 , wherein the capturing of the one or more calibration camera images using the camera comprises:

(a) capturing a preliminary camera image using the camera;

(b) detecting a region of the preliminary camera image covering a pattern of the calibration board;

(c) determining a sharpness score for the region of the preliminary camera image;

(d) until the sharpness score exceeds a sharpness threshold, sending a signal to the camera to adjust a focus of the camera to raise the sharpness score and repeating (a)-(c); and

(d) capturing the one or more calibration camera images using the camera.

8. One or more non-transitory computer readable media storing instructions that in response to being executed by one or more processors, cause a computer system to perform operations, the operations comprising:

determining a target position and orientation for a calibration board with respect to a camera of a vehicle;

detecting a first position and orientation of the calibration board with respect to the camera of the vehicle;

determining instructions for moving the calibration board from the first position and orientation to the target position and orientation;

transmitting the instructions to a device proximate the calibration board;

detecting a second position and orientation of the calibration board with respect to the camera of the vehicle;

determining whether the second position and orientation is within a threshold of matching the target position and orientation; and

in response to determining that the second position and orientation is within the threshold of matching the target position and orientation, capturing one or more calibration camera images using the camera and calibrating one or more sensors of the vehicle using the one or more calibration camera images.

9. The one or more non-transitory computer readable media of claim 8 , wherein:

in response to receiving the instructions, the device is configured to control a machine to automatically reposition the calibration board.

10. The one or more non-transitory computer readable media of claim 8 , wherein:

the device comprises a mobile device; and

in response to receiving the instructions, the mobile device is configured to present corresponding directions to a user to guide the user in manually repositioning the calibration board.

11. The one or more non-transitory computer readable media of claim 10 , wherein the operations further comprise presenting an indication on the mobile device that the second position and orientation of the calibration board is within the threshold of matching the target position and orientation.

12. The one or more non-transitory computer readable media of claim 8 , wherein the capturing of the one or more calibration camera images using the camera comprises:

(a) capturing a preliminary camera image using the camera;

(b) detecting a region of the preliminary camera image covering a pattern of the calibration board;

(c) determining an average value of pixels within the region of the preliminary camera image;

(d) until the average value of the pixels is within an exposure threshold of a predetermined value, sending a signal to the camera to adjust an exposure of the camera to bring the average value of the pixels closer to the predetermined value and repeating (a)-(c); and

(e) capturing the one or more calibration camera images using the camera.

13. The one or more non-transitory computer readable media of claim 12 , wherein the adjusting of the exposure of the camera comprises adjusting a size of an aperture of the camera, adjusting a shutter speed of the camera, or adjusting a sensitivity of a sensor of the camera toward light, or some combination thereof.

14. The one or more non-transitory computer readable media of claim 8 , wherein the capturing of the one or more calibration camera images using the camera comprises:

(a) capturing a preliminary camera image using the camera;

(b) detecting a region of the preliminary camera image covering a pattern of the calibration board;

(c) determining a sharpness score for the region of the preliminary camera image;

(d) until the sharpness score exceeds a sharpness threshold, sending a signal to the camera to adjust a focus of the camera to raise the sharpness score and repeating (a)-(c); and

(d) capturing the one or more calibration camera images using the camera.

15. A computer system comprising:

one or more processors; and

one or more non-transitory computer readable media storing instructions that in response to being executed by the one or more processors, cause the computer system to perform operations, the operations comprising:

determining a target position and orientation for a calibration board with respect to a camera of a vehicle;

detecting a first position and orientation of the calibration board with respect to the camera of the vehicle;

determining instructions for moving the calibration board from the first position and orientation to the target position and orientation;

transmitting the instructions to a device proximate the calibration board;

detecting a second position and orientation of the calibration board with respect to the camera of the vehicle;

determining whether the second position and orientation is within a threshold of matching the target position and orientation; and

in response to determining that the second position and orientation is within the threshold of matching the target position and orientation, capturing one or more calibration camera images using the camera and calibrating one or more sensors of the vehicle using the one or more calibration camera images.

16. The computer system of claim 15 , wherein:

in response to receiving the instructions, the device is configured to control a machine to automatically reposition the calibration board.

17. The computer system of claim 15 , wherein:

the device comprises a mobile device; and

in response to receiving the instructions, the mobile device is configured to present corresponding directions to a user to guide the user in manually repositioning the calibration board.

18. The computer system of claim 17 , wherein the operations further comprise presenting an indication on the mobile device that the second position and orientation of the calibration board is within the threshold of matching the target position and orientation.

19. The computer system of claim 15 , wherein the capturing of the one or more calibration camera images using the camera comprises:

(a) capturing a preliminary camera image using the camera;

(b) detecting a region of the preliminary camera image covering a pattern of the calibration board;

(c) determining an average value of pixels within the region of the preliminary camera image;

(d) until the average value of the pixels is within an exposure threshold of a predetermined value, sending a signal to the camera to adjust an exposure of the camera to bring the average value of the pixels closer to the predetermined value and repeating (a)-(c); and

(e) capturing the one or more calibration camera images using the camera.

20. The computer system of claim 19 , wherein the adjusting of the exposure of the camera comprises adjusting a size of an aperture of the camera, adjusting a shutter speed of the camera, or adjusting a sensitivity of a sensor of the camera toward light, or some combination thereof.

21. The computer system of claim 15 , wherein the capturing of the one or more calibration camera images using the camera comprises:

(a) capturing a preliminary camera image using the camera;

(b) detecting a region of the preliminary camera image covering a pattern of the calibration board;

(c) determining a sharpness score for the region of the preliminary camera image;

(d) until the sharpness score exceeds a sharpness threshold, sending a signal to the camera to adjust a focus of the camera to raise the sharpness score and repeating (a)-(c); and

(d) capturing the one or more calibration camera images using the camera.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2022
From: DEEPMAP INC.
To: NVIDIA CORPORATION
Reel/Frame 061038/0311 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2020
From: HUANG, ZIQIANG; WHEELER, MARK DAMON; YANG, LIN
To: DEEPMAP INC.
Reel/Frame 053245/0400 →
Continuity (2)
Provisional Application 62870897 · Jul 5, 2019
Related Publication 20210004610A1 · Jan 7, 2021
Cited By (4)
US 12,288,363 US 12,380,601 US 12,541,879 US 12,603,986