IP Library › Granted Patent US 10,423,284
Granted Patent B2
US 10,423,284 · App. 15/875,367 · Granted Sep 24, 2019

Electronic apparatus having a sensing unit to input a user command and a method thereof

Inventor: Seungman Kim (McLean, VA)
G06F3/0425G02B13/00G06F3/011G06F3/017G06F3/0304G06F3/0418G06F3/0482G06F3/04817G06F3/04883
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,423,284
App. No.
15/875,367
Granted
Sep 24, 2019
Kind
B2
Abstract

An electronic apparatus includes a housing, a display unit installed in the housing and having a display area to display an image or a menu icon, a sensing unit installed in the housing adjacent to the display unit, having a virtual area to sense an object which is disposed over the display area of the display unit, and having a first camera to photograph and display the virtual area in a photographing mode and a second camera to extract the object from the virtual area and display a sensed image of the extracted object in a sensing mode, and a controller to analyze status of the object according to the sensed image of the object, and to determine the analyzed status of the object as a user input in the sensing mode.

Claims (43)

1. A mobile device, comprising:

a housing having a front side and a rear side;

an input and output installed in the housing, and comprising a panel on the front side of the housing to receive a user input and to display a photographing image or a menu;

a sensing unit installed on the front side of the housing, and having a first camera and a second camera respectively having a first virtual area and a second virtual area which correspond to a display area of the panel; and

a controller installed in the housing, configured to perform a photographing mode to control the first camera to photograph the first virtual area corresponding to the display area of the panel to generate a first photographing image, and to control the display unit to display the first photographing image in the display area of the panel, and configured to perform a sensing mode to control the second camera to photograph the second virtual area including an object disposed to face the display area of the panel to generate a second photographing image representing the object, to determine a state of the object of the second photographing image, and to perform a function of the mobile device according to the menu and the state of the object as an input to the menu,

wherein the second virtual area includes a plurality of sub-objects, one of which has a higher priority than the other one, and the controller is configured to determine a state of the one sub-object with the higher priority as the input to the menu.

2. The mobile device of claim 1 , further comprising:

a memory to store information,

wherein the controller is configured to compare the information and the state of the object and to perform the function of the mobile device according to the menu and the comparison.

3. The mobile device of claim 1 , wherein the controller is configured to analyze the object of the second photographing image with respect to the display area of the panel to sense the state of the object, and to perform the function of the mobile device according to the menu and the analysis.

4. The mobile device of claim 1 , wherein the controller is configured to modify the object of the second photographing image with respect to the display area of the panel to sense the state of the object, and to perform the function of the mobile device according to the menu and the modification.

5. The mobile device of claim 1 , wherein the second camera photographs the object of the second virtual area when the menu is displayed in the display area of the panel.

6. The mobile device of claim 1 , wherein the second camera generates the second photographing image corresponding to the second virtual area, and the controller extracts the object from the second photographing image to sense the state of the object and performs the function of the mobile device according to the menu and the extraction.

7. The mobile device of claim 1 , wherein the controller is configured to display an image corresponding to at least one of the state of the object and the state of the one sub-object over the menu of the display area of the panel.

8. The mobile device of claim 1 , wherein the controller is configured to control the input and output unit to generate an output of the performed function of the mobile device, and the output comprises an image output through the display area of the panel, an audio output through an audio element, or a data output through an interface to an external device.

9. The mobile device of claim 1 , further comprising:

an interface configured to communicate with an external device to transmit or receive data to perform the function of the mobile device,

wherein the controller is configured to perform the function of the mobile device according to the menu of the input and output unit, the state of the object as the user input to the menu, and the data of the external device.

10. The mobile device of claim 1 , wherein:

the controller selects the one sub-object with the higher priority as the object corresponding to the menu; and

the controller performs the function according to at least one of the state of the object and the state of the one sub-object with the higher priority.

11. A method of a mobile device, the method comprising:

providing a housing having a front side and a rear side, an input and output installed in the housing and comprising a panel on the front side of the housing to receive a user input and to display a photographing image or a menu, and a sensing unit installed on the front side of the housing, and having a first camera and a second camera each respectively having a first virtual area and a second virtual area which correspond to a display area of the panel of the input and output unit;

using a controller processor installed in the housing, configured to perform a photographing mode to control the first camera to photograph the first virtual area corresponding to the display area of the panel to generate a first photographing image, and to control the display unit to display the first photographing image in the display area of the panel, and configured to perform a sensing mode to control the second camera to photograph the second virtual area including an object disposed to face the display area of the panel to generate a second photographing image representing the object, to determine a state of the object of the second photographing image, and to perform a function of the mobile device according to the menu and the state of the object as an input to the menu,

wherein the second virtual area includes a plurality of sub-objects, one of which has a higher priority than the other one, and the method further comprises determining a state of the one sub-object with the higher priority as the input to the menu.

12. The method of claim 11 , wherein:

the mobile device further comprises a memory to store information; and

the method further comprises comparing the information and the state of the object and to perform the function of the mobile device according to the menu and the comparison.

13. The method of claim 11 , wherein the method further comprises analyzing the object of the second photographing image to sense the state of the object, and performing the function of the mobile device according to the menu and the analysis.

14. The method of claim 11 , wherein the method further comprises modifying the object of the second photographing image with respect to the display area of the panel to sense the state of the object, and performing the function of the mobile device according to the menu and the modification.

15. The method of claim 11 , wherein the controlling the second camera comprises controlling the second camera to photograph the object of the second virtual area during displaying the menu in the display area of the panel.

16. The method of claim 11 , further comprising:

displaying an image corresponding to at least one of the state of the object and the state of the one sub-object over the menu of the display area of the panel.

17. The method of claim 11 , further comprising:

controlling the input and output unit to generate an output of the performed function of the mobile device,

wherein the output comprises an image output through the display area of the panel, an audio output through an audio element, or a data output through an interface to an external device.

18. The method of claim 11 , further comprising:

an interface configured to communicate with an external device to transmit or receive data to perform a function of the mobile device,

wherein the controller is configured to perform the function of the mobile device according to the menu, the state of the object as the user input to the menu, and the data of the external device.

19. The method of claim 11 , wherein:

the method further comprises selecting the one sub-object with the higher priority as the object corresponding to the menu; and

the method further comprises performing the function according to at least one of the state of the object and the state of the one sub-object with the higher priority.

20. A non-transitory computer-readable medium containing computer-readable codes as a program to perform the method of claim 11 .

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 18, 2026
From: SMARTIP LLC
To: KIMIPS LLC
Reel/Frame 074683/0019 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2024
From: KIM, SEUNGMAN, MR.
To: SMARTIP LLC
Reel/Frame 067773/0897 →
Continuity (5)
Continuation 15440033 · Feb 23, 2017
Continuation 15246302 · Aug 24, 2016
Continuation 14583537 · Dec 26, 2014
Related Publication 20180239494A1 · Aug 23, 2018
Related Publication 20190250770A9 · Aug 15, 2019