Device and method for displaying user interface of virtual input device based on motion recognition

1. A display device comprising:

a camera configured to obtain captured images regarding a user's hand gesture; and

a processor configured to:

in response to a first captured image of the user's hand gesture being obtained, determine a user interface (UI) from among a plurality of UIs respectively corresponding to a type of a virtual input device based on the first captured image of the user's hand gesture, and display the determined UI corresponding to the type of the virtual input device, and

in response to a second captured image of the user's hand gesture being obtained, determine a user command using the UI based on the second captured image,

wherein the processor is further configured to:

recognize a user's finger gesture and a user's gaze to operate the UI,

detect a location of a user's finger on the UI based on the recognized user's finger gesture, and

reset the detected location of the user's finger if the user's gaze is determined to be a non-gaze state,

wherein the captured images include the first captured image and the second captured image,

the UI comprises an image of one among an image of a remote controller, an image of a touch remote controller, an image of a keyboard, and an image of a mouse, and

the user's hand gesture comprises a gesture of operating the virtual input device.

2. The display device of claim 1 , wherein the user's hand gesture comprises a grip gesture of a real input device.

3. The display device of claim 1 , wherein the processor is further configured to recognize a user's facial direction to operate the UI of the virtual input device.

4. The display device of claim 1 , wherein the processor is further configured to control displaying a pointer on the UI of the virtual input device based on the user's finger gesture.

5. The display device of claim 1 , wherein the processor identifies at least one among a movement direction, a movement quantity, and the location of the user's finger, and calculates a size of a pointer on the UI of the virtual input device with respect to an actual finger.

6. The display device of claim 1 , further comprising:

a storage configured to store at least one among a manipulating sound of the virtual input device according to the user's finger gesture, a voice announcing an execution of a function, and a voice announcing a name of a functional button of the virtual input device on which the user's finger is located.

7. The display device of claim 1 , wherein the processor determines the user's gaze as a gaze state or the non-gaze state, with respect to a display, by comparing scatter and concentration of gaze recognition data within a time period with a threshold value.

8. The display device of claim 1 , wherein the processor changes to a voice guiding mode if the user's gaze is determined to be the non-gaze state.

9. The display device of claim 1 , wherein the processor deletes, adds, and modifies a functional button of the UI of the virtual input device according to the user command.

10. A method for displaying a user interface (UI) of a virtual input device, the method comprising:

in response to a first captured image of a user's hand gesture being obtained, determining the UI from among a plurality of UIs respectively corresponding to a type of the virtual input device based on the first captured image of the user's hand gesture, and displaying the determined UI corresponding to the type of the virtual input device, and

in response to a second captured image of the user's hand gesture being obtained, determining a user command using the UI based on the second captured image,

recognizing a user's finger gesture and a user's gaze to operate the UI,

detecting a location of a user's finger on the UI based on the recognized user's finger gesture, and

resetting the detected location of the user's finger if the user's gaze is determined to be a non-gaze state,

wherein the UI comprises an image of one among an image of a remote controller, an image of a touch remote controller, an image of a keyboard, and an image of a mouse, and

the user's hand gesture comprises a gesture of operating the virtual input device.

11. The method of claim 10 , wherein the user's hand gesture comprises a grip gesture with respect to a real input device.

12. The method of claim 10 , further comprising recognizing a user's facial direction to operate the UI of the virtual input device.

13. The method of claim 10 , further comprising displaying a pointer on the UI of the virtual input device based on the recognized user's finger gesture.

14. The method of claim 10 , wherein the recognizing comprises identifying at least one among a movement direction, a movement quantity, and the location of the user's finger and calculating a size of a pointer in the UI of the virtual input device with respect to an actual finger.

15. The method of claim 10 , further comprising:

storing at least one among a manipulating sound of the virtual input device according to the user's finger gesture, a voice announcing an execution of a function, and a voice announcing a name of a functional button of the virtual input device on which the user's finger is located.

16. The method of claim 10 , wherein the recognizing the user's gaze comprises determining a gaze state or the non-gaze state, with respect to a display, by comparing scatter and concentration of gaze recognition data within a time period with a threshold value.

17. The method of claim 10 , further comprising automatically changing to a voice guiding mode if the user's gaze is determined to be the non-gaze state.

18. The method of claim 10 , further comprising at least one among deleting, adding, and modifying a functional button in the UI of the virtual input device according to the user command.