Apparatus, method and recording medium for image stabilization
View Patent ↗An apparatus, a method, and a recording medium for image stabilization are provided. The image stabilization method includes receiving an image that is captured by a camera, detecting a motion of the image, predicting a motion of the image, using a Kalman filter, comparing the detected motion with the predicted motion, and updating a parameter of the Kalman filter based on a result of the comparing.
1. An image stabilization method that is performed by an image stabilization apparatus, the method comprising:
receiving an image that is captured by a camera;
detecting a motion of the image;
predicting a motion of the image, using a Kalman filter; and
increasing a measurement noise variance of the Kalman filter in response to the predicted motion having a fluctuation section, in which an amplitude of the predicted motion is greater than or equal to a first threshold value and the predicted motion fluctuates at a frequency that is greater than or equal to a second threshold.
2. The method of claim 1 , further comprising:
determining an unintentional motion of the image by subtracting the predicted motion from the detected motion; and
compensating for the unintentional motion.
3. The method of claim 1 , wherein the updating the parameter comprising, in response to a difference between the detected motion and the predicted motion being greater than or equal to a third threshold value, determining that over-smoothing occurs and decreasing the measurement noise variance of the Kalman filter to remove an effect of the over-smoothing.
4. The method of claim 1 , further comprising correcting the predicted motion based on the predicted motion and the detected motion.
5. The method of claim 4 , wherein the image is a current frame, and
the method further comprises predicting a motion of a frame next to the current frame based on the corrected predicted motion of the current frame.
6. A non-transitory computer-readable recording medium storing a program to cause a computer to perform the method of claim 1 .
7. An image stabilization method that is performed by an image stabilization apparatus, the method comprising:
receiving an image that is captured by a camera;
detecting a motion of the image;
predicting a motion of the image, using a Kalman filter;
comparing the detected motion with the predicted motion;
updating a parameter of the Kalman filter based on a result of the comparing; and
correcting the predicted motion based on the predicted motion and the detected motion,
wherein the motion is predicted based on an equation:
{
X
^
t
-
=
A
X
^
t
-
1
P
t
-
=
AP
t
-
1
A
T
+
S
w
,
wherein {circumflex over (X)} t − is a motion that is predicted from a current frame, {circumflex over (X)} t−1 is a value of correcting a motion that is predicted from a previous frame, A is a state transition matrix, P t − is an error variance regarding the predicted motion {circumflex over (X)} t − , P t−1 , is an error variance regarding the value {circumflex over (X)} t−1 , and S w is a process noise variance.
8. An image stabilization method that is performed by an image stabilization apparatus, the method comprising:
receiving an image that is captured by a camera;
detecting a motion of the image;
predicting a motion of the image, using a Kalman filter;
comparing the detected motion with the predicted motion;
updating a parameter of the Kalman filter based on a result of the comparing; and
correcting the predicted motion based on the predicted motion and the detected motion,
wherein the predicted motion is corrected based on an equation:
{
K
t
=
P
t
-
C
T
(
CP
t
-
C
T
+
S
z
)
-
1
X
^
t
=
X
^
t
-
+
K
t
(
y
t
-
C
X
^
t
-
)
P
t
=
(
I
2
-
K
t
)
P
t
-
,
wherein k t is a gain of the Kalman filter, C is an observation model matrix, S z is the measurement noise variance of the Kalman filter, {circumflex over (X)} t is a value of correcting a motion {circumflex over (X)} t − that is predicted from a current frame, y t is a motion that is detected from the current frame, and P t is an error variance for the value {circumflex over (X)} t .
9. The method of claim 8 , wherein the updating the parameter comprises updating the measurement noise variance S z .
10. An image stabilization apparatus comprising:
an image receiver configured to receive an image that is captured by a camera; and
at least one processor to implement:
an image corrector configured to detect a motion of the image, and predict a motion of the image, using a Kalman filter; and
a parameter renewer configured to increase a measurement noise variance of the Kalman filter in response to the predicted motion having a fluctuation section, in which an amplitude of the predicted motion is greater than or equal to a first threshold value and the predicted motion fluctuates at a frequency that is greater than or equal to a second threshold.
11. The image stabilization apparatus of claim 10 , wherein the image corrector is further configured to:
determine an unintentional motion of the image by subtracting the predicted motion from the detected motion; and
compensate for the unintentional motion.
12. The image stabilization apparatus of claim 10 , wherein the parameter renewer is further configured to, in response to a difference between the detected motion and the predicted motion being greater than or equal to a third threshold value, determine that over-smoothing occurs and decrease the measurement noise variance of the Kalman filter to remove an effect of the over-smoothing.
13. The image stabilization apparatus of claim 10 , wherein the image corrector is further configured to correct the predicted motion based on the predicted motion and the detected motion.
14. The image stabilization apparatus of claim 13 , wherein the image is a current frame, and
the image corrector is further configured to predict a motion of a frame next to the current frame based on the corrected predicted motion of the current frame.
15. An image stabilization apparatus comprising:
an image receiver configured to receive an image that is captured by a camera; and
at least one processor to implement:
an image corrector configured to detect a motion of the image, and predict a motion of the image, using a Kalman filter; and
a parameter renewer configured to compare the detected motion with the predicted motion, and update a parameter of the Kalman filter based on a result of the comparison, and correct the predicted motion based on the predicted motion and the detected motion,
wherein the motion is predicted based on an equation:
{
X
^
t
-
=
A
X
^
t
-
1
P
t
-
=
AP
t
-
1
A
T
+
S
w
,
wherein {circumflex over (X)} t − is a motion that is predicted from a current frame, {circumflex over (X)} t−1 is a value of correcting a motion that is predicted from a previous frame, A is a state transition matrix, P t − is an error variance for the predicted motion {circumflex over (X)} t − , P t−1 is an error variance for the value {circumflex over (X)} t−1 , and S w is a process noise variance.
16. An image stabilization apparatus comprising:
an image receiver configured to receive an image that is captured by a camera; and
at least one processor to implement:
an image corrector configured to detect a motion of the image, and predict a motion of the image, using a Kalman filter; and
a parameter renewer configured to compare the detected motion with the predicted motion, and update a parameter of the Kalman filter based on a result of the comparison, and correct the predicted motion based on the predicted motion and the detected motion,
wherein the predicted motion is corrected based on an equation:
{
K
t
=
P
t
-
C
T
(
CP
t
-
C
T
+
S
z
)
-
1
X
^
t
=
X
^
t
-
+
K
t
(
y
t
-
C
X
^
t
-
)
P
t
=
(
I
2
-
K
t
)
P
t
-
,
wherein K t is a gain of the Kalman filter, C is an observation model matrix, S z is a measurement noise variance of the Kalman filter, {circumflex over (X)} t is a value of correcting a motion {circumflex over (X)} t − that is predicted from a current frame, y t is a motion that is detected from the current frame, and P t is an error variance for the value {circumflex over (X)} t .
17. The image stabilization apparatus of claim 16 , wherein the parameter renewer is further configured to update the measurement noise variance S z .