IP Library Granted Patent US 9,632,579
Granted Patent B2
US 9,632,579 · App. 14/711,001 · Granted Apr 25, 2017

Device and method of processing image

Inventors: Hong-il Kim (Yongin-si, KR); Tae-hwa Hong (Seoul, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06F3/01G06F3/013G06K9/00G06K9/00604G06K9/46G06K9/4671G06T1/0007G06T1/20G06T7/003G06T13/00H04N5/4448G06K2209/21G06T2207/10004G06T2207/10016G06T2207/30196G06T2207/30232G06T2210/22G06T2210/61G06T2213/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,632,579
App. No.
14/711,001
Granted
Apr 25, 2017
Kind
B2
Abstract

An image processing device includes a first imaging unit configured to capture an image including an object; a display configured to display the image captured by the first imaging unit; a second imaging unit configured to capture a position of eyes of a user; a gaze map generator configured to generate a gaze map including information about a gaze zone on the display according to passage of time based on the position of the eyes; and an image processor configured to generate a motion picture based on the generated gaze map and the captured image.

Claims (56)

1. An image processing device comprising:

a first camera configured to capture an image of an object;

a display configured to display the captured image;

a second camera configured to monitor movements of eyes of a user; and

a processor configured to control to:

generate a gaze map including information about a gaze zone on the display based on the movements of the eyes monitored at a predetermined interval; and

generate a motion picture based on the generated gaze map and the captured image.

2. The image processing device of claim 1 , wherein the captured image is a preview image, and

the processor is further configured to generate a motion picture based on the generated gaze map and the preview image.

3. The image processing device of claim 1 , wherein the processor is further configured to detect a change in the gaze zone for a particular time interval by processing information of the movements of the eyes of the user and generate the gaze map based on the detected change in the gaze zone.

4. The image processing device of claim 1 , wherein the processor is further configured to determine the gaze zone at each time point from the gaze map and generate a motion picture that provides an animation effect of enlarging, reducing, or moving the captured image according to the gaze zone in the captured image.

5. The image processing device of claim 1 , wherein the first camera is further configured to capture a preview image until a still image capturing signal is input, capture a still image when the still image capturing signal is input, and store the captured still image in a storage.

6. The image processing device of claim 5 , wherein the second camera is further configured to continuously capture the movements of the eyes of the user until the still image capturing signal is input.

7. The image processing device of claim 5 , wherein the processor is further configured to determine the gaze zone at each time point from the gaze map, and generate a plurality of cropped images by cropping an area around the gaze zone from the stored still image.

8. The image processing device of claim 7 , wherein the processor is further configured to generate the motion picture by incorporating the generated plurality of cropped images.

9. The image processing device of claim 5 , wherein, when the gaze zone moves within a range equal to or less than a second threshold value for a time over a first threshold value, the processor enlarges the captured still image with respect to the gaze zone and crops the enlarged still image.

10. The image processing device of claim 9 , wherein the gaze zone corresponds to a part of the object included in the stored still image, and

the processor is further configured to enlarge the stored still image at a magnification ratio calculated by using a ratio of an overall size of the object to an overall size of the stored still image.

11. The image processing device of claim 1 , wherein the first camera is further configured to capture a motion picture and store the captured motion picture in a storage, the display is further configured to display a preview image of the motion picture captured by the first camera, and

the processor is further configured to edit the stored motion picture by using the generated gaze map.

12. The image processing device of claim 11 , wherein the processor is further configured to determine the gaze zone at each time point from the gaze map, and crop an area around the gaze zone at a time point corresponding to each frame included in the stored motion picture.

13. The image processing device of claim 12 , wherein the processor is further configured to generate a cropped motion picture by incorporating a plurality of cropped frames.

14. The image processing device of claim 13 , wherein the processor is further configured to store the cropped motion picture in the storage.

15. The image processing device of claim 11 , wherein the processor is further configured to determine importance for each frame of the stored motion picture based on the gaze map, and adjust a reproduction speed of the stored motion picture or emphasize at least one frame based on the determined importance when a user interface for reproducing or editing the stored motion picture is provided.

16. An image processing method comprising:

capturing an image of an object by a first camera;

displaying the captured image on a display;

monitoring movements of eyes of a user by a second camera;

generating a gaze map including information about a gaze zone on the display based on the movements of the eyes monitored at a predetermined interval; and

generating a motion picture based on the generated gaze map and the captured image.

17. The image processing method of claim 16 , wherein the captured image is a preview image, and the generating the motion picture comprises generating the motion picture based on the generated gaze map and the preview image.

18. The image processing method of claim 16 , wherein the generating the motion picture comprises:

determining the gaze zone at each time point from the gaze map; and

generating a plurality of cropped images by cropping an area around the determined gaze zone corresponding to the time point at each time point in the captured image.

19. The image processing method of claim 18 , wherein the generating the motion picture comprises generating the motion picture by incorporating the plurality of generated cropped images.

20. The image processing method of claim 16 , wherein the capturing the image comprises:

capturing a preview image until a still image capturing signal is input;

capturing a still image when the still image capturing signal is input; and

storing the captured still image in a storage.

21. The image processing method of claim 20 , wherein the gaze zone corresponds to a part of the object included in the stored still image, and

the capturing the still image comprises:

enlarging the stored still image at a magnification ratio calculated by using a ratio of an overall size of the object to an overall size of the stored still image when the gaze zone moves within a range equal to or less than a second threshold value for a time over a first threshold value; and

cropping the enlarged still image.

22. The image processing method of claim 16 , wherein the capturing the image comprises capturing a motion picture and storing the captured motion picture in a storage,

the displaying the captured image comprises displaying a preview image of the captured motion picture, and

the generating the motion picture comprises editing the stored motion picture based on the generated gaze map.

23. The image processing method of claim 22 , wherein the editing the stored motion picture comprises:

determining the gaze zone at each time point from the gaze map; and

cropping an area around the determined gaze zone at a time point corresponding to the frame at each frame included in the stored motion picture.

24. The image processing method of claim 23 , wherein the editing the motion picture comprises:

generating a cropped motion picture by incorporating a plurality of cropped frames; and

storing the cropped motion pictures in the storage.

25. The image processing method of claim 22 , wherein the editing the motion picture comprises:

determining importance for each frame of the stored motion picture based on the gaze map; and

adjusting a reproduction speed of the stored motion picture or emphasizing at least one frame based on the determined importance when a user interface for reproducing or editing the stored motion picture is provided.

26. A non-transitory computer readable storage medium storing a program that is executable by a computer to perform the method of claim 16 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2015
From: KIM, HONG-IL; HONG, TAE-HWA
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 035628/0742 →
Priority Claims (1)
KR 10-2014-0057448 · May 13, 2014 · national
Continuity (1)
Related Publication 20150332091A1 · Nov 19, 2015