Method for providing a user interface using three-dimensional gestures and an apparatus using the same
Provided is a method capable of making various modifications to widgets, graphic objects, or images, which are displayed on a display device, according to motions of a plurality of input units such as finger or stylus pen, with the use of a three-dimensional multi-sensor configured to detect the motions of the input units in a space, without touching the display device.
1. A method for controlling a user interface of a mobile device, the method comprising:
controlling display, on a user interface of a mobile device, of a main menu including widgets that each have a depth within a three-dimensional menu space and that each control execution of a specific application on the mobile device;
selecting, from among the widgets in the main menu and in a manner that does not cause execution of a specific application on the mobile device, a widget;
at a time when the widget is selected, detecting, by a controller, a user's gesture related to modifying a depth of the selected widget within the three-dimensional menu space of the main menu; and
based on detecting the user's gesture related to modifying the depth of the selected widget, modifying, on the user interface of the mobile device, the depth of the selected widget within the three-dimensional menu space of the main menu such that, when the mobile device displays the main menu, the selected widget is displayed within the three-dimensional menu space of the main menu at the modified depth, which is different than depths of other widgets within the three-dimensional menu space of the main menu.
2. The method according to claim 1 :
wherein detecting a user's gesture related to modifying a depth of the selected widget comprises detecting a gesture of changing a distance between a user input unit and the mobile device in a space over the selected widget; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises modifying, on the user interface of the mobile device, the depth of the selected widget based on detecting the gesture of changing the distance between the user input unit and the mobile device in the space over the selected widget.
3. The method according to claim 2 :
wherein detecting a gesture of changing a distance between a user input unit and the mobile device in a space over the selected widget comprises detecting a gesture of approaching at least one user input unit to the mobile device or departing the at least one user input unit from the mobile device; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises modifying, on the user interface of the mobile device, the depth of the selected widget based on detecting the gesture of approaching at least one user input unit to the mobile device or departing the at least one user input unit from the mobile device.
4. The method according to claim 1 , further comprising:
detecting a user's gesture of rotating at least one user input unit in a space over the selected widget; and
rotating the selected widget according to the user's gesture of rotating.
5. The method according to claim 1 :
wherein controlling display, on a user interface of a mobile device, of a main menu including widgets that each have a depth within a three-dimensional menu space comprises controlling display, on a user interface of a mobile device, of a main menu including widgets that have different sizes; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises modifying, on the user interface of the mobile device, a size of the selected widget such that, when the mobile device displays the three-dimensional menu space of the main menu, the selected widget is displayed within the three-dimensional menu space of the main menu at the modified size, which is different than sizes of other widgets within the three-dimensional menu space of the main menu.
6. The method according to claim 1 :
wherein controlling display, on a user interface of a mobile device, of a main menu including widgets that each have a depth within a three-dimensional menu space comprises controlling display, on a user interface of a mobile device, of a main menu including widgets that have different shadows; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises modifying, on the user interface of the mobile device, a shadow of the selected widget such that, when the mobile device displays the three-dimensional menu space of the main menu, the selected widget is displayed within the three-dimensional menu space of the main menu with the modified shadow, which is different than shadows of other widgets within the three-dimensional menu space of the main menu.
7. The method of claim 1 :
wherein controlling display, on a user interface of a mobile device, of a main menu including widgets that each have a depth within a three-dimensional menu space comprises controlling display, on a user interface of a mobile device, of a main menu including widgets using at least three discrete levels of depth within the three-dimensional menu space, each of the at least three discrete levels of depth representing a different predetermined depth within the three-dimensional menu space; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises changing, on the user interface of the mobile device, in which of the at least three discrete levels of depth the selected widget is classified such that, when the mobile device displays the three-dimensional menu space of the main menu, the selected widget is displayed within the main menu at the changed level of depth, which is different than levels of depth of other widgets within the three-dimensional menu space of the main menu.
8. A device comprising:
a controller; and
a memory coupled to the controller having stored thereon instructions which, when executed by the controller, causes the controller to perform operations comprising:
controlling display, on a user interface of a mobile device, of a main menu including widgets that each have a depth within a three-dimensional menu space and that each control execution of a specific application on the mobile device;
selecting, from among the widgets in the main menu and in a manner that does not cause execution of a specific application on the mobile device, a widget;
at a time when the widget is selected, detecting a user's gesture related to modifying a depth of the selected widget within the three-dimensional menu space of the main menu; and
based on detecting the user's gesture related to modifying the depth of the selected widget, modifying, on the user interface of the mobile device, the depth of the selected widget within the three-dimensional menu space of the main menu such that, when the mobile device displays the main menu, the selected widget is displayed within the three-dimensional menu space of the main menu at the modified depth, which is different than depths of other widgets within the three-dimensional menu space of the main menu.
9. The device according to claim 8 :
wherein detecting a user's gesture related to modifying a depth of the selected widget comprises detecting a gesture of changing a distance between a user input unit and the mobile device in a space over the selected widget; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises modifying, on the user interface of the mobile device, the depth of the selected widget based on detecting the gesture of changing the distance between the user input unit and the mobile device in the space over the selected widget.
10. The device according to claim 9 :
wherein detecting a gesture of changing a distance between a user input unit and the mobile device in a space over the selected widget comprises detecting a gesture of approaching at least one user input unit to the mobile device or departing the at least one user input unit from the mobile device; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises modifying, on the user interface of the mobile device, the depth of the selected widget based on detecting the gesture of approaching at least one user input unit to the mobile device or departing the at least one user input unit from the mobile device.
11. The device according to claim 8 , wherein the operations further comprise:
detecting a user's gesture of rotating at least one user input unit in a space over the selected widget; and
rotating the selected widget according to the user's gesture of rotating.
12. The device according to claim 8 :
wherein controlling display, on a user interface of a mobile device, of a main menu including widgets that each have a depth within a three-dimensional menu space comprises controlling display, on a user interface of a mobile device, of a main menu including widgets that have different sizes; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises modifying, on the user interface of the mobile device, a size of the selected widget such that, when the mobile device displays the three-dimensional menu space of the main menu, the selected widget is displayed within the three-dimensional menu space of the main menu at the modified size, which is different than sizes of other widgets within the three-dimensional menu space of the main menu.
13. The device according to claim 8 :
wherein controlling display, on a user interface of a mobile device, of a main menu including widgets that each have a depth within a three-dimensional menu space comprises controlling display, on a user interface of a mobile device, of a main menu including widgets that have different shadows; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises modifying, on the user interface of the mobile device, a shadow of the selected widget such that, when the mobile device displays the three-dimensional menu space of the main menu, the selected widget is displayed within the three-dimensional menu space of the main menu with the modified shadow, which is different than shadows of other widgets within the three-dimensional menu space of the main menu.
14. The device according to claim 8 :
wherein controlling display, on a user interface of a mobile device, of a main menu including widgets that each have a depth within a three-dimensional menu space comprises controlling display, on a user interface of a mobile device, of a main menu including widgets using at least three discrete levels of depth within the three-dimensional menu space, each of the at least three discrete levels of depth representing a different predetermined depth within the three-dimensional menu space; and
wherein modifying, on the user interface of the mobile device, the depth of the selected widget comprises changing, on the user interface of the mobile device, in which of the at least three discrete levels of depth the selected widget is classified such that, when the mobile device displays the three-dimensional menu space of the main menu, the selected widget is displayed within the main menu at the changed level of depth, which is different than levels of depth of other widgets within the three-dimensional menu space of the main menu.
15. The method according to claim 1 , wherein modifying, on the user interface of the mobile device, the depth of the selected widget further comprises modifying an area on the user interface of the mobile device at which the selected widget is pressed to cause execution of a specific application on the mobile device associated with the selected widget.
16. The device according to claim 8 , wherein modifying, on the user interface of the mobile device, the depth of the selected widget further comprises modifying an area on the user interface of the mobile device at which the selected widget is pressed to cause execution of a specific application on the mobile device associated with the selected widget.
17. The method according to claim 1 , further comprising:
detecting a user's gesture of rotating at least one user input unit in a space over the selected widget;
determining whether a direction of rotation of the at least one user input unit in the space over the selected widget is in:
a first plane perpendicular to the user interface,
a second plane parallel to the user interface, or
a third plane that is tilted with respect to the user interface such that the third plane is neither perpendicular nor parallel to the user interface;
based on determining that the direction of rotation of the at least one user input unit in the space over the selected widget is in the first plane perpendicular to the user interface, rotating the selected widget, according to the user's gesture of rotating, about a first virtual axis parallel to the user interface;
based on determining that the direction of rotation of the at least one user input unit in the space over the selected widget is in the second plane parallel to the user interface, rotating the selected widget, according to the user's gesture of rotating, about a second virtual axis perpendicular to the user interface; and
based on determining that the direction of rotation of the at last one user input unit in the space over the selected widget is in the third plane that is tilted with respect to the user interface, rotating the selected widget, according to the user's gesture of rotating, about a third virtual axis sloped with respect to the user interface.
18. The method according to claim 1 , wherein selecting the widget comprises selecting a widget that is defined as a rectangular parallelepiped widget, further comprising:
detecting a user's gesture of rotating at least one user input unit in a space over the selected widget; and
in response to detecting the user's gesture of rotating at least one user input unit in the space over the selected widget:
rotating the rectangular parallelepiped widget about an axis to change, from a first side of the rectangular parallelepiped widget, to a second side of the rectangular parallelepiped widget, and
modifying, on the user interface of the mobile device, a display of the selected widget such that, when the mobile device displays the three-dimensional menu space of the main menu, the selected widget is displayed within the three-dimensional menu space of the main menu at the second side of the rectangular parallelepiped widget, rather than the first side of the rectangular parallelepiped widget.
19. The device according to claim 8 , further comprising:
detecting a user's gesture of rotating at least one user input unit in a space over the selected widget;
determining whether a direction of rotation of the at least one user input unit in the space over the selected widget is in:
a first plane perpendicular to the user interface,
a second plane parallel to the user interface, or
a third plane that is tilted with respect to the user interface such that the third plane is neither perpendicular nor parallel to the user interface;
based on determining that the direction of rotation of the at least one user input unit in the space over the selected widget is in the first plane perpendicular to the user interface, rotating the selected widget, according to the user's gesture of rotating, about a first virtual axis parallel to the user interface;
based on determining that the direction of rotation of the at least one user input unit in the space over the selected widget is in the second plane parallel to the user interface, rotating the selected widget, according to the user's gesture of rotating, about a second virtual axis perpendicular to the user interface; and
based on determining that the direction of rotation of the at last one user input unit in the space over the selected widget is in the third plane that is tilted with respect to the user interface, rotating the selected widget, according to the user's gesture of rotating, about a third virtual axis sloped with respect to the user interface.
20. The device according to claim 8 , wherein selecting the widget comprises selecting a widget that is defined as a rectangular parallelepiped widget, further comprising:
detecting a user's gesture of rotating at least one user input unit in a space over the selected widget; and
in response to detecting the user's gesture of rotating at least one user input unit in the space over the selected widget:
rotating the rectangular parallelepiped widget about an axis to change, from a first side of the rectangular parallelepiped widget, to a second side of the rectangular parallelepiped widget, and
modifying, on the user interface of the mobile device, a display of the selected widget such that, when the mobile device displays the three-dimensional menu space of the main menu, the selected widget is displayed within the three-dimensional menu space of the main menu at the second side of the rectangular parallelepiped widget, rather than the first side of the rectangular parallelepiped widget.