IP Library › Granted Patent US 11,991,446
Granted Patent B2
US 11,991,446 · App. 17/851,717 · Granted May 21, 2024

Method of image stabilization and electronic device therefor

Inventor: Wonseok Song (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
H04N23/684H04N23/685H04N23/695
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,991,446
App. No.
17/851,717
Granted
May 21, 2024
Kind
B2
Abstract

A method for operating an electronic device includes: obtaining an image using at least one of a first camera having a first FOV or a second camera comprising a reflective member and having a second FOV narrower than the first FOV; obtaining, from a motion sensor, first movement information of the electronic device, which is shaken to correspond to the obtaining of the image; obtaining information on movement of the reflective member of the second camera according to at least one of an optical image stabilizer (OIS) correction operation or an object tracking operation during at least a portion of a period in which the image is obtained; obtaining output information by performing an image stabilization operation with respect to the image based on at least one of the first movement information or the information on the movement of the reflective member; and outputting an image corrected by correcting distortion of the image based on a distortion variation resulting from the movement of the reflective member and the output information.

Claims (71)

1. An electronic device comprising:

a camera module comprising a first camera having a first field of view (FOV) and a second camera having a second FOV narrower than the first FOV, the second FOV is positioned in the first FOV;

a motion sensor; and

a processor electrically connected to the camera module and the motion sensor,

wherein the second camera comprises:

at least one lens;

a reflective member configured to rotate relative to the at least one lens; and

an image sensor configured to generate an electrical signal based on light incident through the reflective member and the at least one lens, and

wherein the processor is configured to:

obtain an image using the camera module;

obtain, from the motion sensor, first movement information of the electronic device which is shaken, to correspond to the obtaining of the image;

obtain information on movement of the reflective member of the second camera according to at least one of an optical image stabilizer (OIS) correction operation or an object tracking operation during at least a portion of a period in which the image is obtained;

obtain output information by performing an image stabilization operation with respect to the image, based on at least one of the first movement information or the information on the movement of the reflective member; and

output an image corrected by correcting distortion of the image, based on a distortion variation resulting from the movement of the reflective member and the output information.

2. The electronic device of claim 1 , wherein the information on the movement of the reflective member comprises:

second movement information corresponding to the movement of the reflective member of the second camera, which is made as the OIS correction operation is performed; and

third movement information corresponding to the movement of the reflective member of the second camera which is made, as the object tracking operation is performed.

3. The electronic device of claim 1 , wherein the processor is further configured to:

obtain a plurality of first movement information from the motion sensor while at least partially obtaining the image; and

obtain the first movement information, which corresponds to the obtained image, among the plurality of first movement information.

4. The electronic device of claim 3 , wherein the processor is further configured to:

receive a synchronization signal from at least one of the first camera or the second camera, at a time point at which the at least one of the first camera or the second camera obtains the image.

5. The electronic device of claim 4 , wherein the processor is further configured to:

obtain, based on the synchronization signal, synchronized first movement information corresponding to the time point, at which the at least one of the first camera or the second camera obtains the image, among the plurality of first movement information.

6. The electronic device of claim 2 , wherein the processor is further configured to:

perform the image stabilization operation, based on the first movement information and the at least one of the second movement information or the third movement information, according to whether the object tracking operation is performed.

7. The electronic device of claim 6 , wherein the processor is further configured to:

perform the image stabilization operation, based on movement information which is obtained by removing the second movement information and the third movement information from the first movement information, when the object tracking operation is performed.

8. The electronic device of claim 1 , wherein the processor is further configured to:

obtain the distortion variation of the image distorted as the reflective member of the second camera moves; and

perform the correcting distortion of the image based on the distortion variation, and

wherein the output information is used as an input for the correcting the distortion of the image.

9. The electronic device of claim 8 , wherein the distortion variation indicates a distortion degree of pixels included in the image distorted,

wherein the processor is further configured to:

obtain the distortion variation based on the information on the movement of the reflective member, and

wherein at least a portion of the image distorted is corrected based on the distortion variation.

10. The electronic device of claim 1 , wherein the processor is further configured to:

move the second FOV of the second camera within the first FOV of the first camera by rotating the reflective member.

11. A method for operating an electronic device, the method comprising:

obtaining an image using at least one of a first camera having a first field of view (FOV) or a second camera comprising a reflective member and having a second FOV narrower than the first FOV, the second FOV is positioned in the first FOV;

obtaining, from a motion sensor, first movement information of the electronic device, which is shaken to correspond to the obtaining of the image;

obtaining information on movement of the reflective member of the second camera according to at least one of an optical image stabilizer (OIS) correction operation or an object tracking operation during at least a portion of a period in which the image is obtained;

obtaining output information by performing an image stabilization operation with respect to the image based on at least one of the first movement information or the information on the movement of the reflective member; and

outputting an image corrected by correcting distortion of the image based on a distortion variation resulting from the movement of the reflective member and the output information.

12. The method of claim 11 , wherein the information on the movement of the reflective member comprises:

second movement information corresponding to the movement of the reflective member of the second camera, which is made as the OIS correction operation is performed; and

third movement information corresponding to the movement of the reflective member of the second camera, which is made as the object tracking operation is performed.

13. The method of claim 11 , wherein the obtaining of the first movement information comprises:

obtaining a plurality of first movement information from the motion sensor, while at least partially obtaining the image; and

obtaining the first movement information, which corresponds to the obtained image, among the plurality of first movement information.

14. The method of claim 13 , further comprising:

receiving a synchronization signal from at least one of the first camera or the second camera, at a time point at which the at least one of the first camera or the second camera obtains the image.

15. The method of claim 14 , wherein the obtaining of the first movement information comprises:

obtaining, based on the synchronization signal, synchronized first movement information corresponding to the time point at which the at least one of the first camera or the second camera obtains the image, among the plurality of first movement information.

16. The method of claim 12 , wherein the obtaining of the output information comprises:

performing the image stabilization operation, based on the first movement information and the at least one of the second movement information or the third movement information, according to whether the object tracking operation is performed.

17. The method of claim 16 , further comprising:

performing the image stabilization operation, based on movement information which is obtained by removing the second movement information and the third movement information from the first movement information, when the object tracking operation is performed.

18. The method of claim 11 , further comprising:

obtaining the distortion variation of the image distorted as the reflective member of the second camera moves;

wherein the correcting the distortion of the image is performed based on the distortion variation, and

wherein the output information is used as an input for the correcting the distortion of the image.

19. The method of claim 18 , wherein the distortion variation indicates a distortion degree of pixels included in the image,

wherein the distortion variation is obtained based on the information on the movement of the reflective member, and

wherein at least a portion of the image distorted is corrected based on the distortion variation.

20. The method of claim 11 , wherein the second camera comprises:

a lens assembly; and

an image sensor configured to generate an electrical signal using light incident through the reflective member and the lens assembly,

wherein the reflective member is configured to rotate relative to the image sensor and the lens assembly, and

wherein the method further comprises:

moving the second FOV of the second camera within the first FOV of the first camera by rotating the reflective member.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2022
From: SONG, WONSEOK
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 060448/0700 →
Priority Claims (1)
KR 10-2021-0084714 · Jun 29, 2021 · national
Continuity (2)
Continuation PCTKR2022003409 · Mar 11, 2022
Related Publication 20220417405A1 · Dec 29, 2022
Cited By (1)
US 12,574,451