Image processing apparatus, image processing method, and storage medium
An image processing apparatus for calibrating an image capture apparatus that includes an optical system and an imaging sensor, the image processing apparatus including: an obtaining unit configured to obtain an image captured by the image capture apparatus; a detection unit configured to detect whether vibration occurred in the image capture apparatus; and an updating unit configured to in a case where the detection unit detects that the vibration occurred, update calibration information to correct the image captured by the image capture apparatus based on the captured image obtained by the obtaining unit.
1 . An image processing apparatus for calibrating an image capture apparatus that includes an optical system and an imaging sensor, the image processing apparatus comprising:
at least one processor or circuit configured to function as a plurality of units comprising:
(1) an obtaining unit configured to obtain an image captured by the image capture apparatus;
(2) a detection unit configured to detect whether predetermined vibration occurred in the image capture apparatus;
(3) a correction unit configured to perform a process of correction on the image captured by the image capture apparatus based on calibration information; and
(4) an updating unit configured to update, asynchronously with the process of the correction performed by the correction unit, the calibration information based on the image obtained by the obtaining unit, in a case where the detection unit detects that the predetermined vibration has stopped.
2 . The image processing apparatus according to claim 1 , wherein the updating unit updates the calibration information based on the captured image after the detection unit detects that the predetermined vibration has stopped.
3 . The image processing apparatus according to claim 1 , wherein the updating unit updates the calibration information in a case where a predetermined condition is satisfied.
4 . The image processing apparatus according to claim 3 , wherein the updating unit updates the calibration information in a case where an evaluation value of the calibration information before the calibration information is updated for an image captured by the image capture apparatus after the predetermined vibration occurred satisfies the predetermined condition.
5 . The image processing apparatus according to claim 4 , wherein the case where the predetermined condition is satisfied includes a case where the image capture apparatus is subjected to misalignment of the optical system.
6 . The image processing apparatus according to claim 2 , wherein the updating unit determines an evaluation value of the captured image applied to a temporary calibration value after the predetermined vibration occurs and sets the temporary calibration value as the updated calibration information in a case where the determined evaluation value reaches a predetermined value.
7 . The image processing apparatus according to claim 6 , wherein the evaluation value is a value corresponding to the sum of squares of errors in a least squares method.
8 . The image processing apparatus according to claim 1 , wherein the correction unit, based on the calibration information, corrects the influence of optical axis misalignment in the optical system on the image captured by the image capture apparatus.
9 . The image processing apparatus according to claim 1 , wherein for a frame before the updating unit updates the calibration information, the correction unit corrects the influence based on the calibration information before the calibration information is updated, and
wherein for a frame after the updating unit updates the calibration information, the correction unit corrects the influence based on the calibration information after the calibration information is updated.
10 . The image processing apparatus according to claim 1 , wherein the image capture apparatus includes a sensor,
wherein the detection unit obtains a value of the sensor, and
wherein in a case where the obtained value of the sensor exceeds a predetermined value, the detection unit detects that the predetermined vibration occurred.
11 . The image processing apparatus according to claim 10 , wherein the sensor is an accelerometer.
12 . The image processing apparatus according to claim 1 , wherein the optical system includes a first optical system and a second optical system.
13 . An image processing method to calibrate an image capture apparatus that includes an optical system and an imaging sensor, the method comprising:
obtaining an image captured by the image capture apparatus;
detecting whether predetermined vibration occurred in the image capture apparatus;
performing a process of correction on the image captured by the image capture apparatus based on calibration information; and
updating, asynchronously with the process of the correction, the calibration information based on the obtained image, in a case where the detecting detects that the predetermined vibration has stopped.
14 . A non-transitory computer-readable storage medium storing a program which causes a computer to perform an image processing method to calibrate an image capture apparatus that includes an optical system and an imaging sensor, the image processing method comprising:
obtaining an image captured by the image capture apparatus;
detecting whether predetermined vibration occurred in the image capture apparatus;
performing a process of correction on the image captured by the image capture apparatus based on calibration information; and
updating, asynchronously with the process of the correction, the calibration information based on the obtained image, in a case where the detecting detects that the predetermined vibration has stopped.