Apparatus and method for estimating optical image stabilization motion
An electronic device is disclosed. Moreover, various embodiment found through the disclosure are possible. The electronic device may include a camera module including an optical image stabilizer, a motion sensor, a processor electrically connected to the camera module and the motion sensor. The processor may be configured to obtain first motion information corresponding to shaking of the electronic device through the motion sensor, while at least partially obtaining an image frame, determine second motion information, which corresponds to motion of a lens or an image sensor included in the camera module, from the first motion information, using attribute information and a frequency pass filter of the optical image stabilizer, and perform digital image correction, based on the first motion information and the second motion information.
1. An electronic device comprising:
a camera module comprising a lens, an image sensor, and an optical image stabilizer controlling a position of the image sensor and the lens;
a motion sensor; and
a processor electrically connected to the camera module and the motion sensor,
wherein the processor is configured to:
obtain an image frame with a first frequency, which is subject to optical image correction by the optical image stabilizer, wherein the optical image correction corresponding to a shaking of the electronic device,
obtain a first motion value corresponding to the shaking of the electronic device through the motion sensor with a second frequency, while at least partially obtaining the image frame,
determine a second motion value, which corresponds to motion of the lens or the image sensor, by applying the first motion value to a high frequency pass filter of the optical image stabilizer,
determine a third motion value by removing the second motion value from the first motion value through the motion sensor, and
perform digital image correction for at least a portion of the image frame, based on the third motion value, and
wherein the second frequency is higher than the first frequency.
2. The electronic device of claim 1 , wherein the first motion value includes:
a range of an angle, at which the lens or the image sensor moves by the optical image stabilizer.
3. The electronic device of claim 1 , wherein the processor is further configured to:
correct a first line of a plurality of lines included in the image frame using a first value, which corresponds to the first line, of the second motion value, and
correct a second line of the plurality of lines using a second value, which corresponds to the second line, of the second motion value.
4. The electronic device of claim 1 , wherein the processor is further configured to:
determine whether the second frequency of the first motion value is less than a first threshold value which is specified,
perform the digital image correction based on the first motion value and the second motion value, when the second frequency of the first motion value is equal to or greater than the first threshold value, and
perform the digital image correction based on at least one of the first motion value or a fourth motion value, which represents motion of at least one feature point in the image frame, when the second frequency of the first motion value is less than the first threshold value.
5. The electronic device of claim 4 , wherein the processor is further configured to:
perform the digital image correction based on the fourth motion value, when the second frequency of the first motion value is less than a second threshold value which is specified.
6. The electronic device of claim 5 , wherein the processor is further configured to:
perform the digital image correction based on the first motion value or based on the first motion value and the fourth motion value, when the at least one feature point is not identified in the image frame within a specified time, the image frame includes a repeated pattern, or the image frame includes local motion.
7. The electronic device of claim 1 , wherein the motion sensor is embedded in the camera module or attached to an outer portion of the camera module.
8. A method of an electronic device including a motion sensor and a camera module comprising a lens, an image sensor, and an optical image stabilizer controlling a position of the image sensor and the lens, the method comprising:
obtaining an image frame subject to optical image correction by the optical image stabilizer with a first frequency, wherein the optical image correction corresponding to a shaking of the electronic device;
obtaining a first motion value corresponding to the shaking of the electronic device with a second frequency, while at least partially obtaining the image frame;
determining a second motion value, which corresponds to motion of the lens or the image sensor, by applying the first motion value to a high frequency pass filter of the optical image stabilizer;
determining a third motion value by removing the second motion value from the first motion value; and
performing digital image correction for at least a portion of the image frame, based on multiple pieces of the third motion value,
wherein the second frequency is higher than the first frequency.
9. The method of claim 8 , wherein the first motion value includes:
a range of an angle at which the lens or the image sensor moves by the optical image stabilizer.
10. The method of claim 8 , wherein the obtaining the image frame subject to the optical image correction includes:
correcting a first line of a plurality of lines included in the image frame using a first value, which corresponds to the first line, of the second motion value, and
correcting a second line of the plurality of lines using the second motion value, which corresponds to the second line, of the second motion value.
11. The method of claim 8 , further comprising:
determining whether the second frequency of the first motion value is less than a first threshold value which is specified;
performing the digital image correction based on the first motion value and the second motion value, when the second frequency of the first motion value is equal to or greater than the first threshold value; and
performing the digital image correction based on at least one of the first motion value or a fourth motion value, which represents motion of at least one feature point in the image frame, when the second frequency of the first motion value is less than the first threshold value.
12. The method of claim 11 , further comprising:
performing the digital image correction based on the fourth motion value, when the second frequency of the first motion value is less than a second threshold value which is specified.
13. The method of claim 12 , further comprising:
performing the digital image correction based on the first motion value or based on the first motion value and the fourth motion value, when the at least one feature point is not identified in the image frame within a specified time, the image frame includes a repeated pattern, or the image frame includes local motion.