IP Library Granted Patent US 11,061,480
Granted Patent B2
US 11,061,480 · App. 16/806,421 · Granted Jul 13, 2021

Apparatus, method and recording medium for controlling user interface using input image

Inventors: Jin-Yong Kim (Yongin-si, KR); Ji-Young Kang (Suwon-si, KR); Dae-Sung Kim (Seoul, KR); Seok-Tae Kim (Daejeon, KR); Bo-Young Lee (Seoul, KR); Seung-Kyung Lim (Seoul, KR); Jin-Young Jeon (Seoul, KR)
Assignee: Samsung Electronics Co., Ltd.
G06F3/017G06F3/005G06F3/011G06F3/0304G06K9/00302G06K9/00335H04N5/232H04N5/23216H04N5/23218H04N5/23219H04N5/232127G06F2203/0381H04N5/23293
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,061,480
App. No.
16/806,421
Granted
Jul 13, 2021
Kind
B2
Abstract

A method of controlling a user interface using an input image is provided. The method includes storing operation executing information of each of one or more gesture forms according to each of a plurality of functions, detecting a gesture form from the input image, and identifying the operation executing information mapped on the detected gesture form to execute an operation according to a function which is currently operated.

Claims (42)

1. A method of controlling a user interface of an electronic device, the method comprising:

storing operation information mapped to each of one or more gesture forms and each of a plurality of applications;

obtain an input image through one or more camera of the electronic device while a first application is being executed;

detecting a gesture form from the input image;

identifying a first operation information from among the stored operation information that is mapped to the detected gesture form and the first application; and

perform an operation corresponding to the first operation information.

2. An electronic device comprising:

one or more cameras;

a storage; and

at least one processor configured to:

store, in the storage, operation information mapped to each of one or more gesture forms and each of a plurality of applications,

obtain an input image through the one or more cameras while a first application is being executed,

detect a gesture form from the input image,

identify first operation information from among the stored operation information that is mapped to the detected gesture form and the first application, and

perform an operation corresponding to the first operation information.

3. The electronic device as claimed in claim 2 , wherein the at least one processor is further configured to:

extract a contour from the input image when the gesture form is detected from the input image,

determine whether the extracted contour has a form matched with the gesture form, and

determine the matched form as the gesture form and detects the gesture form if it is determined that the extracted contour has the form matched with the one or more gesture forms.

4. The electronic device as claimed in claim 2 , wherein the at least one processor is further configured to tag a predetermined metadata corresponding to the detected gesture form to the input image when the first operation information mapped on the detected gesture form is identified and the operation of the first operation information is executed.

5. The electronic device as claimed in claim 2 ,

wherein the first application is a camera application, and

wherein the at least one processor is further configured to:

detect a predetermined pointing position of a predetermined pointing gesture form from the input image if the detected gesture form is the predetermined pointing gesture form, and

execute an auto focusing operation of the camera function by a reference of the detected pointing position.

6. The electronic device as claimed in claim 2 , further comprising:

a display,

wherein a predetermined texture corresponding to the detected gesture form is applied to the input image displayed on a preview screen of the display, when the first operation information mapped on the detected gesture form is identified and the operation of the first operation information is executed.

7. The electronic device as claimed in claim 2 , further comprising:

a display,

wherein the at least one processor is further configured to:

detect at least one object located in a pointing direction of a pointing gesture form from the input image displayed on a preview screen of the display if the detected gesture form is a predetermined pointing gesture form, and

apply a predetermined texture corresponding to the detected gesture form to the detected object.

8. The electronic device as claimed in claim 2 , further comprising:

a display,

wherein the at least one processor is further configured to display the input image and a predetermined visible effect corresponding to the detected gesture form of the input image on the display, when the first operation information mapped on the detected gesture form is identified and the operation of the first operation information is executed.

9. The electronic device as claimed in claim 8 , wherein the at least one processor is further configured to:

apply a face recognition function to the input image so as to detect at least one portion of a face from the input image, and

display the predetermined visible effect on the detected face portion corresponding to a predetermined face portion in correspondence to the detected gesture form.

10. The electronic device as claimed in claim 8 , further comprising:

a microphone,

wherein the at least one processor is further configured to change at least one of a type, a color or a shape of the visible effect according to recognition of a predetermined voice instruction input through the microphone.

Priority Claims (1)
KR 10-2013-0108697 · Sep 10, 2013 · national
Continuity (3)
Continuation 15896751 · Feb 14, 2018
Continuation 14478399 · Sep 5, 2014
Related Publication 20200201446A1 · Jun 25, 2020