Image processing technique for generating more appropriate motion vectors for a captured image that is distorted
A generation unit generates N distortion corrected images (N≥2) by performing coordinate conversion on each of N partial areas of a first captured image, using a conversion parameter that depends on a position of the partial area, and generates N distortion corrected images by performing coordinate conversion on each of corresponding N partial areas of a second captured image, using a conversion parameter that depends on a position of the partial area. A detection unit detects N motion vectors by comparing each of the N distortion corrected images generated from the first captured image with the corresponding distortion corrected image generated from the second captured image. A conversion unit generates N converted motion vectors by performing inverse coordinate conversion on each of the N motion vectors using a conversion parameter that depends on a position of the corresponding partial area.
1. An image processing apparatus comprising:
one or more processors configured to execute one or more programs so as to function as units comprising:
(1) an acquisition unit configured to acquire a first captured image and a second captured image generated by photo-electrically converting a subject image formed by an optical system;
(2) a generation unit configured to generate N distortion corrected images, N being an integer greater than or equal to 2, by performing coordinate conversion on each of N partial areas of the first captured image, using a conversion parameter that depends on a position of the partial area, and to generate N distortion corrected images by performing coordinate conversion on each of corresponding N partial areas of the second captured image, using a conversion parameter that depends on a position of the partial area;
(3) a detection unit configured to detect N motion vectors by comparing each of the N distortion corrected images generated from the first captured image with the corresponding distortion corrected image generated from the second captured image; and
(4) a conversion unit configured to generate N converted motion vectors by performing inverse coordinate conversion on each of the N motion vectors, using a conversion parameter that depends on a position of the corresponding partial area.
2. The image processing apparatus according to claim 1 , wherein the one or more processors further function as units comprising:
an output unit configured to output a first image corresponding to a first partial area of the first captured image; and
an identification unit configured to identify a second partial area corresponding to a movement destination of the first partial area in the second captured image based on the N converted motion vectors,
wherein the output unit outputs a second image corresponding to the second partial area of the second captured image.
3. The image processing apparatus according to claim 2 , wherein the output unit generates a first distortion corrected image as the first image by performing coordinate conversion on the first partial area, using a conversion parameter that depends on a position of the first partial area, and generates a second distortion corrected image as the second image by performing coordinate conversion on the second partial area, using a conversion parameter that depends on a position of the second partial area.
4. The image processing apparatus according to claim 2 , wherein the identification unit calculates the movement destination of the first partial area based on an affine transformation of the N converted motion vectors.
5. The image processing apparatus according to claim 2 , wherein each of the N partial areas of the first captured image is smaller than the first partial area, and
wherein each of the N partial areas of the second captured image is smaller than the second partial area.
6. The image processing apparatus according to claim 1 , wherein the first captured image and the second captured image are circular images corresponding to a hemispherical field of view, and
wherein the conversion parameter include an azimuth angle and a zenith angle.
7. An image capturing apparatus comprising:
(A) one or more processors configured to execute one or more programs so as to function as units comprising:
(1) an acquisition unit configured to acquire a first captured image and a second captured image generated by photo-electrically converting a subject image formed by an optical system;
(2) a generation unit configured to generate N distortion corrected images, being an integer greater than or equal 2, to by performing coordinate conversion on each of N partial areas of the first captured image, using a conversion parameter that depends on a position of the partial area, and to generate N distortion corrected images by performing coordinate conversion on each of corresponding N partial areas of the second captured image, using a conversion parameter that depends on a position of the partial area;
(3) a detection unit configured to detect N motion vectors by comparing each of the N distortion corrected images generated from the first captured image with the corresponding distortion corrected image generated from the second captured image; and
(4) a conversion unit configured to generate N converted motion vectors by performing inverse coordinate conversion on each of the N motion vectors, using a conversion parameter that depends on a position of the corresponding partial area; and
(B) an image capturing unit configured to photo-electrically convert a subject image formed by the optical system.
8. A control method of an image processing apparatus, comprising:
acquiring a first captured image and a second captured image generated by photo-electrically converting a subject image formed by an optical system;
generating N distortion corrected images, N being an integer greater than or equal to 2, by performing coordinate conversion on each of N partial areas of the first captured image, using a conversion parameter that depends on a position of the partial area;
generating N distortion corrected images by performing coordinate conversion on each of corresponding N partial areas of the second captured image, using a conversion parameter that depends on a position of the partial area;
detecting N motion vectors by comparing each of the N distortion corrected images generated from the first captured image with the corresponding distortion corrected image generated from the second captured image; and
generating N converted motion vectors by performing inverse coordinate conversion on each of the N motion vectors, using a conversion parameter that depends on a position of the corresponding partial area.
9. A non-transitory computer-readable storage medium which stores a program for causing a computer to execute a control method comprising:
acquiring a first captured image and a second captured image generated by photo-electrically converting a subject image formed by an optical system;
generating N distortion corrected images, N being an integer greater than or equal to 2, by performing coordinate conversion on each of N partial areas of the first captured image, using a conversion parameter that depends on a position of the partial area;
generating N distortion corrected images by performing coordinate conversion on each of corresponding N partial areas of the second captured image, using a conversion parameter that depends on a position of the partial area;
detecting N motion vectors by comparing each of the N distortion corrected images generated from the first captured image with the corresponding distortion corrected image generated from the second captured image; and
generating N converted motion vectors by performing inverse coordinate conversion on each of the N motion vectors, using a conversion parameter that depends on a position of the corresponding partial area.