Extrinsic calibration method for cameras of an on-board system for formation of stereo images
View Patent ↗Disclosed is a method to calibrate an on-board stereo system. The method includes correlating the depth deviation of a point of a scene observed by the system with respect to a supposedly planar scene and the corresponding yaw deviation between the cameras of the system, then deducing therefrom a yaw calibration correction for the cameras. The comparison between the scene as observed and as expected consists in determining, via spatio-temporal filtering, a depth deviation between the observed depth of at least one point of a planar scene image formed in the image plane of first camera, as positioned in the coordinate system of the other camera, and the expected depth of this point projected onto the planar scene from the first camera, then in determining the yaw calibration deviation between the cameras as a function of the deviation in depth averaged over a sufficient set of points.
1. An extrinsic calibration method for a first ( 11 ) and a second ( 12 ) camera of a stereoscopic system for formation of stereo images, said system on-board a motor vehicle ( 1 ), comprising the steps of:
an initial calibration step ( 31 ) of acquiring roll, pitch and yaw stereo extrinsic and mono intrinsic calibrations for the first and second cameras, including a yaw calibration of the cameras;
acquiring successive stereoscopic images of a scene in a field of vision by reconstructing three-dimensional images calibrated using the initial calibration step, each stereoscopic image of the successive stereoscopic images being reconstructed from left and right images produced simultaneously by the first and second cameras ( 11 , 12 ), respectively, and digitally stored as pixels;
filtering the reconstructed stereoscopic images;
comparing, using spatio-temporal filtering with a predetermined number of points, a profile of the scene as observed via each reconstructed image ( 10 ) and a linear profile of the scene ( 120 ) as expected, and generating an angular yaw calibration deviation (ΔL) therefrom;
correcting the yaw calibration of the cameras ( 11 , 12 ) on the basis of the generated yaw calibration deviation (ΔL); and
iterating the steps of filtering, comparing, and correcting until the generated yaw calibration deviation (ΔL) is zero.
2. The calibration method as claimed in claim 1 , wherein the comparing of the profile of the scene as observed and the linear profile of the scene as expected includes the sub-steps of
determining a field depth deviation (Δa, ΔZb) between a field depth of at least one point (P 1 a , P 1 b ) of the scene as observed and a field depth of at least one point (P 2 a , P 2 b ) of the scene as expected ( 120 ),
and then determining the yaw calibration deviation (ΔL) between the first and second cameras ( 11 , 12 ) as a function of the field depth deviation averaged over a determined set of points.
3. The calibration method as claimed in claim 2 , wherein the observed depth of field of an image point (Pi) of a planar scene formed by the first camera ( 11 ) is determined as a function of a disparity between the images produced by the first and second cameras ( 11 , 12 ).
4. The calibration method as claimed in claim 3 , wherein a distance between said first camera ( 11 ) and the image point (Pi) is equal to the product of a focal length (f) of the first camera ( 11 ) and a distance (b) between the first and second cameras ( 11 , 12 ) divided by the disparity between the images of the first and second cameras.
5. The calibration method as claimed in claim 2 , wherein an expected depth (Zs) is determined by a transition matrix between a coordinate system (O 1 X 1 Y 1 Z 1 ) of said first camera ( 11 ) and a coordinate system (OXYZ) of the vehicle ( 1 ).
6. The calibration method as claimed in claim 2 , wherein the yaw calibration deviation (ΔL) is, for each image point (Pi), proportional to the field depth deviation (ΔZa, ΔZb) and inversely proportional to the square of the expected depth (Zs).
7. The calibration method as claimed in claim 2 , wherein the yaw calibration deviation (ΔL) between the first ( 11 ) camera and the second ( 12 ) camera corresponds to a convergence, or a divergence, respectively, between the optical axes (O 1 Z 1 , O 2 Z 2 ) of the cameras ( 11 , 12 ), depending on whether the profile of the observed scene ( 130 ) is generally inclined with a generally positive slope ( 13 a ) or a generally negative slope ( 13 b ), respectively, with respect to a generally zero slope of the linear profile of the scene ( 120 ) as expected.
8. The calibration method as claimed in claim 2 ,
wherein a yaw angle elementary variation dL between the first ( 11 ) camera and the second camera ( 12 ), as a function of an elementary depth deviation dZ, between a same point as observed (P 1 a , P 1 b ) and as expected (P 2 a , P 2 b ) of the observed profile of the scene ( 130 ; 13 a , 13 b ) and of the expected linear profile of the scene ( 120 ), respectively, is given by the formula
d
L
=
-
dZ
*
b
(
Z
2
*
(
1
+
Un
2
)
)
,
where
“b” is a distance between the cameras,
“Z” is the expected depth of the point (P 2 a , P 2 b ) in the linear profile scene ( 120 ), and
“Un” is a normalized transverse coordinate of an image point (Pi) corresponding to the point observed in a two-dimensional image coordinate system of the first camera, where
Un
=
(
Ui
-
U
0
)
f
,
“Ui” being a transverse coordinate of the image point in an image plane of an image of the first camera ( 11 ),
“U 0 ” being a centered coordinate in said image, and
“f” being a focal length of said first camera ( 11 ).
9. The calibration method as claimed in claim 3 , wherein an expected depth (Zs) is determined by a transition matrix between a coordinate system (O 1 X 1 Y 1 Z 1 ) of said first camera ( 11 ) and a coordinate system (OXYZ) of the vehicle ( 1 ).
10. The calibration method as claimed in claim 4 , wherein an expected depth (Zs) is determined by a transition matrix between a coordinate system (O 1 X 1 Y 1 Z 1 ) of said first camera ( 11 ) and a coordinate system (OXYZ) of the vehicle ( 1 ).
11. The calibration method as claimed in claim 3 , wherein the yaw calibration deviation (ΔL) is, for each image point (Pi), proportional to the field depth deviation (ΔZa, ΔZb) and inversely proportional to the square of the expected depth (Zs).
12. The calibration method as claimed in claim 4 , wherein the yaw calibration deviation (ΔL) is, for each image point (Pi), proportional to the field depth deviation (ΔZa, ΔZb) and inversely proportional to the square of the expected depth (Zs).
13. The calibration method as claimed in claim 5 , wherein the yaw calibration deviation (ΔL) is, for each image point (Pi), proportional to the field depth deviation (ΔZa, ΔZb) and inversely proportional to the square of the expected depth (Zs).
14. The calibration method as claimed in claim 3 , wherein the yaw calibration deviation (ΔL) between the first ( 11 ) camera and the second ( 12 ) camera corresponds to a convergence, or a divergence, respectively, between the optical axes (O 1 Z 1 , O 2 Z 2 ) of the cameras ( 11 , 12 ), depending on whether the profile of the observed scene ( 130 ) is generally inclined with a generally positive slope ( 13 a ) or a generally negative slope ( 13 b ), respectively, with respect to a generally zero slope of the linear profile of the scene ( 120 ) as expected.
15. The calibration method as claimed in claim 4 , wherein the yaw calibration deviation (ΔL) between the first ( 11 ) camera and the second ( 12 ) camera corresponds to a convergence, or a divergence, respectively, between the optical axes (O 1 Z 1 , 02 Z 2 ) of the cameras ( 11 , 12 ), depending on whether the profile of the observed scene ( 130 ) is generally inclined with a generally positive slope ( 13 a ) or a generally negative slope ( 13 b ), respectively, with respect to a generally zero slope of the linear profile of the scene ( 120 ) as expected.
16. The calibration method as claimed in claim 5 , wherein the yaw calibration deviation (ΔL) between the first ( 11 ) camera and the second ( 12 ) camera corresponds to a convergence, or a divergence, respectively, between the optical axes (O 1 Z 1 , O 2 Z 2 ) of the cameras ( 11 , 12 ), depending on whether the profile of the observed scene ( 130 ) is generally inclined with a generally positive slope ( 13 a ) or a generally negative slope ( 13 b ), respectively, with respect to a generally zero slope of the linear profile of the scene ( 120 ) as expected.
17. The calibration method as claimed in claim 6 , wherein the yaw calibration deviation (ΔL) between the first ( 11 ) camera and the second ( 12 ) camera corresponds to a convergence, or a divergence, respectively, between the optical axes (o 1 Z 16 , O 2 Z 2 ) of the cameras ( 11 , 12 ), depending on whether the profile of the observed scene ( 130 ) is generally inclined with a generally positive slope ( 13 a ) or a generally negative slope ( 13 b ), respectively, with respect to a generally zero slope of the linear profile of the scene ( 120 ) as expected.
18. The calibration method as claimed in claim 3 ,
wherein a yaw angle elementary variation dL between the first ( 11 ) camera and the second camera ( 12 ), as a function of an elementary depth deviation dZ, between a same point as observed (P 1 a , P 1 b ) and as expected (P 2 a , P 2 b ) of the observed profile of the scene ( 130 ; 13 a , 13 b ) and of the expected linear profile of the scene ( 120 ), respectively, is given by the formula
d
L
=
-
dZ
*
b
(
Z
2
*
(
1
+
Un
2
)
)
,
where
“b” is a distance between the cameras,
“Z” is the expected depth of the point (P 2 a , P 2 b ) in the linear profile scene ( 120 ), and
“Un” is a normalized transverse coordinate of an image point (Pi) corresponding to the point observed in a two-dimensional image coordinate system of the first camera, where
Un
=
(
Ui
-
U
0
)
f
,
“Ui” being a transverse coordinate of the image point in an image plane of an image of the first camera ( 11 ),
“U 0 ” being a centered coordinate in said image, and
“f” being a focal length of said first camera ( 11 ).
19. The calibration method as claimed in claim 4 ,
wherein a yaw angle elementary variation dL between the first ( 11 ) camera and the second camera ( 12 ), as a function of an elementary depth deviation dZ, between a same point as observed (P 1 a , P 1 b ) and as expected (P 2 a , P 2 b ) of the observed profile of the scene ( 130 ; 13 a , 13 b ) and of the expected linear profile of the scene ( 120 ), respectively, is given by the formula
d
L
=
-
dZ
*
b
(
Z
2
*
(
1
+
Un
2
)
)
,
where
“b” is a distance between the cameras,
“Z” is the expected depth of the point (P 2 a , P 2 b ) in the linear profile scene ( 120 ), and
“Un” is a normalized transverse coordinate of an image point (Pi) corresponding to the point observed in a two-dimensional image coordinate system of the first camera, where
Un
=
(
Ui
-
U
0
)
f
,
“Ui” being a transverse coordinate of the image point in an image plane of an image of the first camera ( 11 ),
“U 0 ” being a centered coordinate in said image, and
“f” being a focal length of said first camera ( 11 ).
20. The calibration method as claimed in claim 5 ,
wherein a yaw angle elementary variation dL between the first ( 11 ) camera and the second camera ( 12 ), as a function of an elementary depth deviation dZ, between a same point as observed (P 1 a , P 1 b ) and as expected (P 2 a , P 2 b ) of the observed profile of the scene ( 130 ; 13 a , 13 b ) and of the expected linear profile of the scene ( 120 ), respectively, is given by the formula
d
L
=
-
dZ
*
b
(
Z
2
*
(
1
+
Un
2
)
)
,
where
“b” is a distance between the cameras,
“Z” is the expected depth of the point (P 2 a , P 2 b ) in the linear profile scene ( 120 ), and
“Un” is a normalized transverse coordinate of an image point (Pi) corresponding to the point observed in a two-dimensional image coordinate system of the first camera, where
Un
=
(
Ui
-
U
0
)
f
,
“Ui” being a transverse coordinate of the image point in an image plane of an image of the first camera ( 11 ),
“U 0 ” being a centered coordinate in said image, and
“f” being a focal length of said first camera ( 11 ).