Method for calibrating a vehicle cabin camera

1 . A method for calibrating a vehicle cabin camera, said camera having: a pitch angle X around a vehicle transverse axis, a yaw angle Y around a vertical axis and a roll angle Z around a vehicle longitudinal axis; and a field of view capturing a plurality of vehicle cabin features which are symmetric about said vehicle longitudinal axis across a range of pitch, yaw and roll positions of said camera, said camera being incorporated within a vehicle rear view mirror, the method comprising:

obtaining a set of 3D points representing edges from a CAD model of a vehicle interior;

acquiring an image of the vehicle by said camera;

choosing a candidate rotation in each of X, Y, and Z directions;

performing a rotation of said 3D points around a Y- and Z-axis according to a YZ rotation candidate;

performing a rotation of said 3D points around X-axis with a number of candidate X rotations;

selecting a set of points in a set of edges acquired from the CAD model of a vehicle interior;

projecting each of said points onto a 3D unit circle in accordance with a camera projection model;

determining a candidate rotation around the vertical axis and the vehicle longitudinal axis;

computing rotation values for the vertical axis and the vehicle longitudinal axis based on the determined rotation;

rotating the selected set of points based on the rotation values;

calibrating the camera based on the rotated set of points around a transverse axis;

generating a gradient map corresponding to the acquired image by said camera;

selecting said set of points from points within said gradient map having a gradient value greater than a threshold;

for each selected point from within said gradient map, comparing the gradient value for the selected point with respective gradient values of one or more points of locations of said rotated set of points closest to said selected point to provide a difference; and

aggregating said differences to obtain a measure of a match for the candidate rotation.

2 . The method of claim 1 , wherein the camera is chosen from any of a visible wavelength RGB camera, a Bayer array type camera with sensitive RGB-IR or RGB-W pixels, a dedicated IR camera, a bolometer, or an event camera.

3 . The method of claim 1 further comprising performing calibration around said transverse axis by:

obtaining one or more 3D vehicle coordinate space locations for respective distinctive cabin features visible across a range of pitch, yaw and roll positions of said camera;

rotating said 3D vehicle coordinate space locations according to the determined rotation;

for each of one or more rotations of a set of candidate pitch rotations;

further rotating the rotated 3D vehicle coordinate space locations with the rotation;

mapping the further rotated 3D vehicle coordinate space locations into an image plane to provide a set of transformed 3D vehicle coordinate space locations; and

selecting a candidate rotation which provides a best match between the set of transformed 3D vehicle coordinate space locations and locations of corresponding distinctive cabin features in an image acquired by the camera.

4 . The method of claim 1 , further comprising a step for determining a rotation around said transverse axis, being performed either: before said steps of claim 1 or after said steps of claim 1 .

5 . The method of claim 1 comprising calibrating at least one other camera incorporated within said vehicle rear view mirror using said camera calibration.

6 . The method of claim 1 further comprising determining a location of said camera in 3D vehicle coordinate space according to said calibration and knowledge of a spatial relationship between said camera location and a ball joint mount for said rear view mirror.

7 . A computer program product comprising a non-statutory computer readable medium on which instructions are stored which when executed on a processor of a vehicle system are configured to perform a method of calibrating a vehicle cabin camera, said camera having: a pitch angle X around a vehicle transverse axis, a yaw angle Y around a vertical axis and a roll angle Z around a vehicle longitudinal axis; and a field of view capturing a plurality of vehicle cabin features which are symmetric about said vehicle longitudinal axis across a range of pitch, yaw and roll positions of said camera, said camera being incorporated within a vehicle rear view mirror, the method comprising:

obtaining a set of 3D points representing edges from a CAD model of a vehicle interior;

acquiring an image of the vehicle by said camera;

choosing a candidate rotation in each of X, Y, and Z directions;

performing a rotation of said set of 3D points around a Y- and Z-axis according to a YZ rotation candidate;

performing a rotation of said set of 3D points around X-axis with a number of candidate X rotations;

selecting a set of points in a set of edges acquired from the CAD model of a vehicle interior;

projecting each of said points onto a 3D unit circle in accordance with a camera projection model;

determining a candidate rotation around the vertical axis and the vehicle longitudinal axis;

computing rotation values for the vertical axis and the vehicle longitudinal axis based on the determined rotation;

rotating the selected set of points based on the rotation values;

calibrating the camera based on the rotated set of points around a transverse axis;

generating a gradient map corresponding to the acquired image by said camera;

selecting said set of points from points within said gradient map having a gradient value greater than a threshold;

for each selected point from within said gradient map, comparing the gradient value for the selected point with respective gradient values of one or more points of locations of said rotated set of points closest to said selected point to provide a difference; and

aggregating said differences to obtain a measure of a match for the candidate rotation.

8 . A system for calibrating a vehicle cabin camera, said camera having: a pitch angle X around a vehicle transverse axis, a yaw angle Y around a vertical axis and a roll angle Z around a vehicle longitudinal axis; and a field of view capturing a plurality of vehicle cabin features which are symmetric about said vehicle longitudinal axis across a range of pitch, yaw and roll positions of said camera, said camera being incorporated within a vehicle rear view mirror, the system comprising:

one or more memories;

one or more processors in communication with the one or more memories, the one or more processors configured to:

obtain a set of 3D points representing edges from a CAD model of a vehicle interior;

acquire an image of the vehicle by said camera;

choose a candidate rotation in each of X, Y, and Z directions;

perform a rotation of said set of 3D points around a Y- and Z-axis according to a YZ rotation candidate;

perform a rotation of said set of 3D points around X-axis with a number of candidate X rotations;

select a set of points in a set of edges acquired from the CAD model of a vehicle interior;

project each of said points onto a 3D unit circle in accordance with a camera projection model;

determine a candidate rotation around the vertical axis and the vehicle longitudinal axis;

compute rotation values for the vertical axis and the vehicle longitudinal axis based on the determined rotation;

rotate the selected set of points based on the rotation values; calibrate the camera based on the rotated set of points around a transverse axis;

generate a gradient map corresponding to the acquired image by said camera;

select said set of points from points within said gradient map having a gradient value greater than a threshold;

for each selected point from within said gradient map, compare the gradient value for the selected point with respective gradient values of one or more points of locations of said rotated set of points closest to said selected point to provide a difference; and

aggregate said differences to obtain a measure of a match for the candidate rotation.

9 . The system of claim 8 , wherein the camera is chosen from any of a visible wavelength RGB camera, a Bayer array type camera with sensitive RGB-IR or RGB-W pixels, a dedicated IR camera, a bolometer, or an event camera.

10 . The system of claim 8 , wherein the one or more processors are further configured to perform calibration around said transverse axis by:

obtaining one or more 3D vehicle coordinate space locations for respective distinctive cabin features visible across a range of pitch, yaw and roll positions of said camera;

rotating said 3D vehicle coordinate space locations according to the determined rotation;

for each of one or more rotations of a set of candidate pitch rotations;

further rotating the rotated 3D vehicle coordinate space locations with the rotation; and

mapping the further rotated 3D vehicle coordinate space locations into an image plane to provide a set of transformed 3D vehicle coordinate space locations; and

selecting a candidate rotation which provides a best match between the set of transformed 3D vehicle coordinate space locations and locations of corresponding distinctive cabin features in an image acquired by the camera.

11 . The system of claim 10 , wherein one or more processors are further configured to determine a set of candidate yaw and roll rotations spaced apart from one another at a first angular resolution and find an improved candidate yaw and roll rotation at said first angular resolution within the selected candidate rotation.

12 . The system of claim 8 , wherein the one or more processors are further configured to perform determining a rotation around said transverse axis, being performed either before steps of claim 9 or after said steps of claim 8 .

13 . The system of claim 8 , wherein the one or more processors are further configured to:

calibrate at least one other camera incorporated within said vehicle rear view mirror using said camera calibration.

14 . The system of claim 8 , wherein the one or more processors are further configured to determine a location of said camera in 3D vehicle coordinate space according to said calibration and knowledge of a spatial relationship between said camera location and a ball joint mount for said rear view mirror.