IP Library Granted Patent US 10,696,240
Granted Patent B2
US 10,696,240 · App. 15/207,667 · Granted Jun 30, 2020

Drive-by calibration from static targets

Inventors: Lingjun Gao (Uckfield, GB); Heba Khafagy (Chiswick, GB); Dave Wibberley (Brighton, GB)
Assignee: Continental Automotive GmbH
B60R11/04G06T7/13G06T7/246G06T7/50G06T7/73G06T7/74G06T7/80B60R2300/307B60R2300/402G06T2207/10016G06T2207/30204G06T2207/30244G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,696,240
App. No.
15/207,667
Granted
Jun 30, 2020
Kind
B2
Abstract

A method for deriving extrinsic camera parameters of a vehicle camera. Calibration markers are provided on a flat ground surface and the vehicle is driven past the calibration markers. Marker boundaries are detected and matched to stored pre-determined shape parameters and a marker shape is identified. At least one extrinsic parameter of the camera is derived using the tracked positions of identified marker shape in the video sequence captured while vehicle is moving, wherein the extrinsic parameter is selected from mounting positions and a rotation is selected from both horizontal axis and vertical axis of a vehicle coordinate system.

Claims (53)

1. A method for deriving extrinsic camera parameters of at least one vehicle camera, comprising:

providing separate calibration markers having respective perimeters in a shape that is common between the separate calibration markers, the separate calibration markers manually placed at random locations and random orientation angles with respect to the respective perimeters on a ground surface relative to each other and relative to the placement of the vehicle,

storing, by a computer readable memory of the vehicle, pre-determined shape parameters corresponding to each of the separate calibration markers,

receiving, by a processor of the vehicle, image data from the at least one vehicle camera, the image data comprising a video sequence, the video sequence comprising calibration marker image data captured over a period of time while the vehicle is driven towards and over at least one of the separate calibration markers,

detecting, by the processor of the vehicle, marker boundaries in the image data,

matching, by the processor of the vehicle, the detected marker boundaries to the pre-determined shape parameters,

identifying, by the processor of the vehicle, the separate calibration markers,

tracking, by the processor of the vehicle, the identified separate calibration markers in the video sequence captured over the period of time while the vehicle is driven towards and over at least one of the separate calibration markers,

deriving, by the processor of the vehicle, at least one extrinsic parameter of the at least one vehicle camera using the positions of the identified separate calibration markers in the video sequence captured over the period of time while the vehicle is driven towards and over at least one of the separate calibration markers, the at least one extrinsic parameter of the at least one vehicle camera including at least one rotation about at least one axis, and

calibrating the at least one vehicle camera by adjusting the at least one rotation to attain an optimum rotation about the at least one axis.

2. The method according to claim 1 , wherein the marker boundaries comprise a boundary of an internal pattern of a calibration marker image.

3. The method according to claim 1 , comprising:

deriving a perspective foreshortening of a calibration marker image from a matching of detected marker boundaries to the pre-determined shape parameters, and

deriving an orientation of the at least one vehicle camera with respect to the ground plane from the derived perspective foreshortening.

4. The method according to claim 1 comprising:

deriving an orientation of the at least one vehicle camera with respect to the ground plane from the at least one extrinsic parameter, and

deriving a rotation around a first horizontal axis from the derived orientation of the camera with respect to the ground plane.

5. The method according to claim 1 comprising:

deriving an orientation of the at least one vehicle camera with respect to the ground plane from the at least one extrinsic parameter, and

deriving a rotation around a second horizontal axis from the derived orientation of the camera with respect to the ground plane.

6. The method according to claim 1 comprising:

deriving a magnitude of a shape parameter of a calibration marker image,

comparing the magnitude of the shape parameter with a magnitude of a corresponding stored shape parameter, and

deriving a mounting height of the at least one camera from the comparison of the magnitudes.

7. The method according to claim 1 comprising:

deriving a magnitude of a shape parameter of a first calibration marker image from image data of a first vehicle camera,

deriving a distance between the first vehicle camera and the corresponding calibration marker,

deriving a magnitude of a shape parameter of a second calibration marker image from image data of a second vehicle camera,

the first calibration marker image and the second calibration marker image corresponding to the same calibration marker,

deriving a distance between the second vehicle camera and the corresponding calibration marker,

deriving a position of the corresponding calibration marker on the ground surface with respect to a vehicle coordinate system using the first distance and the second distance,

deriving a rotation of the first vehicle camera around a vertical axis from the derived position of the corresponding calibration marker, and

deriving a rotation of the second vehicle camera around the vertical axis from the derived position of the corresponding calibration marker.

8. The method according to claim 7 , comprising deriving a rotation of the first vehicle camera around a first horizontal axis from the derived position of the corresponding calibration marker.

9. The method according to claim 7 , comprising deriving a rotation of the first vehicle camera around a second horizontal axis from the derived position of the corresponding calibration marker.

10. The method according to claim 1 , wherein the image data comprises two or more calibration marker images, comprising:

deriving a camera orientation of the at least one camera for each of the two or more calibration marker images, and

computing an average camera orientation as a weighted average of the derived camera orientations of the at least one camera.

11. The method according to claim 1 , wherein the calibration markers are selected from one or more painted surfaces and one or more portable boards.

12. A non-transitory computer readable storage medium, storing a computer program for the execution of a method according to claim 1 .

13. A vehicle computation unit, comprising:

a computer readable memory, the computer readable memory comprising stored pre-determined shape parameters of a set of separate calibration markers,

wherein the calibration markers have respective perimeters in a shape that is common between the separate calibration markers, the separate calibration markers are manually placed at random locations and random orientation angles with respect to the respective perimeters on a ground surface relative to each other and relative to the placement of the vehicle;

a processor; and

a connection for receiving image data from one or more vehicle cameras, the image data comprising a video sequence, the video sequence comprising calibration marker image data captured over a period of time while the vehicle is driven towards and over at least one of the separate calibration markers,

wherein the processor is configured to:

detect marker boundaries in the image data,

match the detected marker boundaries to the pre-determined shape parameters,

identify each of the separate calibration markers,

track the identified separate calibration markers in the video sequence captured over the period of time while the vehicle is driven towards and over at least one of the separate calibration markers,

derive at least one extrinsic parameter of the at least one vehicle camera using the positions of the identified separate calibration markers in the video sequence captured over the period of time while the vehicle is driven towards and over at least one of the separate calibration markers, the at least one extrinsic parameter of the at least one vehicle camera including at least one rotation about at least one axis, and

calibrate the at least one vehicle camera by adjusting the at least one rotation to attain an optimum rotation about the at least one axis.

14. The method according to claim 8 , comprising deriving a rotation of the first vehicle camera around a second horizontal axis from the derived position of the corresponding calibration marker.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2025
From: CONTINENTAL AUTOMOTIVE GMBH
To: CONTINENTAL AUTONOMOUS MOBILITY GERMANY GMBH
Reel/Frame 070438/0643 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2016
From: GAO, LINGJUN; KHAFAGY, HEBA; WIBBERLEY, DAVE
To: CONTINENTAL AUTOMOTIVE GMBH
Reel/Frame 039923/0058 →
Priority Claims (1)
EP 15178814 · Jul 29, 2015 · regional
Continuity (1)
Related Publication 20170032526A1 · Feb 2, 2017