IP Library Granted Patent US 11,538,192
Granted Patent B1
US 11,538,192 · App. 17/398,965 · Granted Dec 27, 2022

Method for calibrating a vehicle cabin camera

Inventors: Piotr Stec (Galway, IE); Petronel Bigioi (Galway, IE)
Assignee: FotoNation Limited
G06T7/80B60R1/12B60R11/04G06T7/13B60R2001/1253B60R2011/0033G06T2207/30268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,538,192
App. No.
17/398,965
Granted
Dec 27, 2022
Kind
B1
Abstract

A method for calibrating a vehicle cabin camera having: a pitch, yaw and roll angle; and a field of view capturing vehicle cabin features which are symmetric about a vehicle longitudinal axis comprises: selecting points from within an image of the vehicle cabin and projecting the points onto a 3D unit circle in accordance with a camera projection model. For each of one or more rotations of a set of candidate yaw and roll rotations, the method comprises: rotating the projected points with the rotation; flipping the rotated points about a pitch axis; counter-rotating the projected points with an inverse of the rotation; and mapping the counter-rotated points back into an image plane to provide a set of transformed points. A candidate rotation which provides a best match between the set of transformed points and the locations of the selected points in the image plane is selected.

Claims (34)

1. A method for calibrating a vehicle cabin camera, said camera having: a pitch angle X around a vehicle transverse axis, a yaw angle Y around a vertical axis and a roll angle Z around a vehicle longitudinal axis; and a field of view capturing a plurality of vehicle cabin features which are symmetric about said vehicle longitudinal axis across a range of pitch, yaw and roll positions of said camera, the method comprising:

selecting a set of points from within an image of the vehicle cabin acquired by said cabin camera,

projecting each of said points onto a 3D unit circle in accordance with a camera projection model,

for each of one or more rotations R YZ of a set of candidate yaw and roll rotations,

rotating the projected points with the rotation R YZ ;

flipping the rotated points about an X axis;

counter-rotating the projected points with an inverse of the rotation R YZ ; and

mapping the counter-rotated points back into an image plane to provide a set of transformed point locations; and

selecting a candidate rotation which provides a best match between the set of transformed point locations and the locations of the selected set of points in the image plane as the YZ rotation of the camera.

2. The method of claim 1 further comprising:

detecting a set of edges within said image of the vehicle cabin, and

selecting said set of points from within said edge locations.

3. The method of claim 1 wherein said camera is incorporated within a vehicle rear view mirror.

4. The method of claim 1 further comprising performing calibration around said transverse axis by:

obtaining one or more 3D vehicle coordinate space locations for respective distinctive cabin features visible across a range of pitch, yaw and roll positions of said camera;

rotating said coordinate space locations according to the selected rotation R YZ ;

for each of one or more rotations R X of a set of candidate pitch rotations,

further rotating the rotated 3D vehicle coordinate space locations with the rotation R X ; and

mapping the further rotated locations into the image plane to provide a set of transformed locations; and

selecting a candidate rotation R X which provides a best match between the set of transformed 3D vehicle coordinate space locations and the locations of corresponding distinctive cabin features in an image acquired by the camera.

5. The method of claim 3 further comprising performing calibration around said transverse axis by:

obtaining an acceleration measurement from an accelerometer incorporated within one of said camera or said mirror;

obtaining an acceleration measurement from an accelerometer incorporated in said vehicle outside of said mirror; and

combining said acceleration measurements to determine a rotation R X around said transverse axis.

6. The method of claim 5 wherein said steps for determining a rotation R X around said transverse axis are performed either: before said steps of claim 1 or after said steps of claim 1 .

7. The method of claim 1 wherein said set of candidate yaw and roll rotations are spaced apart from one another at a first angular resolution and wherein said method further comprises finding an improved candidate yaw and roll rotation within said angular resolution of said selected candidate rotation.

8. The method of claim 1 comprising:

generating a gradient map corresponding to said acquired image;

selecting said set of points from points within said gradient map having a gradient greater than a threshold;

for each selected point from within said gradient map, comparing the gradient value for the selected point with respective gradient values of one or more points of said transformed point locations closest to said selected point to provide a difference; and

aggregating said differences to obtain a measure of the match for a candidate rotation.

9. The method of claim 3 comprising calibrating at least one other camera incorporated within said vehicle rear view mirror using said camera calibration.

10. The method of claim 3 further comprising determining a location of said camera in 3D vehicle coordinate space according to said calibration and knowledge of a spatial relationship between said camera location and a ball joint mount for said rear view mirror.

11. A computer program product comprising a computer readable medium on which instructions are stored which when executed on a processor of a vehicle system are configured to perform the steps of claim 1 .

Assignments (2)
CHANGE OF NAME Recorded May 19, 2025
From: FOTONATION LIMITED
To: TOBII TECHNOLOGIES LIMITED
Reel/Frame 071292/0964 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 10, 2021
From: STEC, PIOTR; BIGIOI, PETRONEL
To: FOTONATION LIMITED
Reel/Frame 057140/0349 →