Image warping correction in forming 360 degree panoramic images
View Patent ↗A method for creating a 360 degree panoramic image from multiple images includes (1) computing a gross rotation error ΔR between a first image and a calculated first image rotated to be stitched to a last image, and (2) spreading the gross rotation error ΔR to each pixel on the panoramic image. Spreading the gross rotation error ΔR includes (1) computing a rotation angle θ 0 and rotational axis n 0 from the gross rotational error ΔR, (2) determining an angle α of each pixel, and (3) determining a compensation matrix R c for each pixel using the following formula: R c (α)=R(α/2πθ 0 ). Spreading the gross rotation error ΔR further includes (4) tracing a pixel on the panoramic image to a camera optical center of the images to form a first ray, (5) determining a second ray originating from the camera optical center that would be rotated by the compensation matrix R c to coincide with the first ray, (6) tracing the second ray to a second pixel on one of the images, and (7) painting the first pixel with color values of the second pixel.
1. A method for creating a 360 degree panoramic image from a plurality of images, comprising:
arranging the images around a camera optical center and projecting the arranged images onto a cylindrical surface to generate the 360 degree panoramic image;
determining a gross rotation error ΔR between a first image and a last image from the arranged images, wherein the gross rotation error ΔR is a rotation necessary to rotate the first image to match the last image; and
dividing the gross rotation error ΔR among all columns of pixels in the 360 degree panoramic image, comprising:
for each column, determining a compensation matrix R c for the column using a rotation angle θ 0 and rotation axis n 0 of the gross rotation error ΔR, and a rotation angle α of the column; and
for each pixel in each column, replacing the pixel with a corresponding pixel identified by the compensation matrix R c for the column.
2. The method of claim 1 , wherein said determining a compensation matrix R c for the column comprises using the following formula:
R
c
(
α
)
=
[
n
1
2
+
(
1
-
n
1
2
)
cos
(
α
2
π
θ
0
)
n
1
n
2
(
1
-
cos
θ
)
-
n
3
sin
(
α
2
π
θ
0
)
n
1
n
3
(
1
-
cos
θ
)
+
n
2
sin
(
α
2
π
θ
0
)
n
1
n
2
(
1
-
cos
(
α
2
π
θ
0
)
)
+
n
3
sin
(
α
2
π
θ
0
)
n
2
2
+
(
1
-
n
2
2
)
cos
(
α
2
π
θ
0
)
n
2
n
3
(
1
-
cos
θ
)
-
n
1
sin
(
α
2
π
θ
0
)
n
1
n
3
(
1
-
cos
(
α
2
π
θ
0
)
)
-
n
2
sin
(
α
2
π
θ
0
)
n
2
n
3
(
1
-
cos
(
α
2
π
θ
0
)
)
+
n
1
sin
(
α
2
π
θ
0
)
n
3
2
+
(
1
-
n
3
2
)
cos
(
α
2
π
θ
0
)
]
,
where the rotation axis n 0 is [n 1 , n 2 , n 3 ] T .
3. The method of claim 2 , further comprising using a plurality of relative rotation matrices between adjacent images from the arranged images to calculated the rotation angle θ 0 and the rotation axis n 0 .
4. The method of claim 2 , wherein said replacing the pixel with a corresponding pixel comprises:
tracing the pixel in the column on the 360 degree panoramic image to the camera optical center of the arranged images to form a first ray;
determining a second ray originating from the camera optical center that would be rotated by the compensation matrix R c to coincide with the first ray; and
tracing the second ray to an intersection with the corresponding pixel on one of the arranged images; and
painting the pixel with color values of the corresponding pixel.
5. A method for creating a 360 degree panoramic image from a plurality of images, comprising:
determining a gross rotation error ΔR between a first image and a last image, wherein the gross rotation error ΔR is a rotation necessary to rotate the first image to match the last image;
determining a rotation angle θ 0 and a rotation axis n 0 for the gross rotation error ΔR, comprising using a plurality of relative rotation matrices between adjacent images from the plurality of images to calculated the rotation angle θ 0 and the rotation axis n 0 ;
determining a rotation angle α of a column on the 360 degree panoramic image;
determining a compensation matrix R c for each column using the following formula:
R
c
(
α
)
=
[
n
1
2
+
(
1
-
n
1
2
)
cos
(
α
2
π
θ
0
)
n
1
n
2
(
1
-
cos
θ
)
-
n
3
sin
(
α
2
π
θ
0
)
n
1
n
3
(
1
-
cos
θ
)
+
n
2
sin
(
α
2
π
θ
0
)
n
1
n
2
(
1
-
cos
(
α
2
π
θ
0
)
)
+
n
3
sin
(
α
2
π
θ
0
)
n
2
2
+
(
1
-
n
2
2
)
cos
(
α
2
π
θ
0
)
n
2
n
3
(
1
-
cos
θ
)
-
n
1
sin
(
α
2
π
θ
0
)
n
1
n
3
(
1
-
cos
(
α
2
π
θ
0
)
)
-
n
2
sin
(
α
2
π
θ
0
)
n
2
n
3
(
1
-
cos
(
α
2
π
θ
0
)
)
+
n
1
sin
(
α
2
π
θ
0
)
n
3
2
+
(
1
-
n
3
2
)
cos
(
α
2
π
θ
0
)
]
,
where the rotation axis n 0 is [n 1 , n 2 , n 3 ] T ;
tracing a first pixel in the column to a camera optical center of the plurality of images to form a first ray;
determining a second ray originating from the camera optical center that would be rotated by the compensation matrix R c to coincide with the first ray;
tracing the second ray to an intersection with a second pixel on one of the plurality of images;
painting the first pixel with color values of the second pixel;
repeating said tracing a first pixel, said determining a second ray, said tracing the second ray, and said painting for other pixels in the column the compensation matrix R c ; and
repeating said determining a rotation angle α, said determining a compensation matrix R c , said tracing a first pixel, said determining a second ray, said tracing the second ray, and said painting for other columns on the 360 degree panoramic image.