IP Library › Granted Patent US 9,292,112
Granted Patent B2
US 9,292,112 · App. 13/543,419 · Granted Mar 22, 2016

Multimodal interface

Inventors: Sriganesh Madhvanath (Karnataka, IN); Dustin Freeman (Ontario, CA)
Assignee: Hewlett-Packard Development Company, L.P.
G06F3/038G06F3/04817G06F3/04883G06F2203/0381
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,292,112
App. No.
13/543,419
Granted
Mar 22, 2016
Kind
B2
Abstract

Provided is a multimodal graphical user interface. The multimodal graphical user interface includes a menu with at least one menu item, wherein the at least one menu item is displayed as command name along with a unique hand shape, wherein the at least one menu item is configured to receive a combination of cursor and selection gesture input.

Claims (27)

1. A system comprising:

a processor; and

a memory storing program instructions, which when executed by the processor cause the system to generate a multimodal graphical user interface, the multimodal graphical user interface comprising a display of a menu of a plurality of different selectable menu items, wherein the display of each of the plurality of different selectable menu items includes a command name and a unique hand shape corresponding to the command name,

wherein one of the plurality of different selectable menu items is selected when the menu item receives a combination of a cursor input and a selection gesture input, wherein the selection gesture input includes an air-tap gesture.

2. The system of claim 1 , wherein each of the plurality of different selectable menu items is selected in response to a speech input.

3. The system of claim 2 , wherein the speech input includes the command name.

4. The system of claim 1 , wherein each of the plurality of different selectable menu items is selected in response to a hand shape gesture input.

5. The system of claim 4 , wherein the hand shape gesture input corresponds to the unique hand shape in the menu.

6. The system of claim 1 , wherein the selection gesture input further includes a snap of fingers.

7. The system of claim 1 , wherein the one of the plurality of different selectable menu items further includes at least one sub-menu item, wherein the at least one sub-menu item is displayed as another command name and another unique hand shape, wherein the at least one sub-menu item is selected in response to a combination of the cursor input and the air-tap gesture input.

8. The system of claim 7 , wherein the at least one sub-menu item is selected in response to a speech input.

9. The system of claim 8 , wherein the speech input includes the command name.

10. The system of claim 7 , wherein the at least one sub-menu item is selected in response to a hand shape gesture input.

11. The system of claim 10 , wherein the hand shape gesture input corresponds to the another unique hand shape in the sub-menu item.

12. The system of claim 1 , wherein the plurality of different selectable menu items is displayed in at least one of the following layouts: radial, vertical and horizontal.

13. A method for allowing a user to interface with a computing device, comprising:

displaying, with a display device, a multimodal graphical user interface, wherein the multimodal graphical user interface comprises a display of a menu of a plurality of different selectable menu items, wherein the display of each of the plurality of different selectable menu items includes a command name and a unique hand shape corresponding to the command name;

accepting, by the computing device, a combination of a cursor input and an air-tap gesture input for selecting one of the plurality of different selectable menu items, wherein the cursor input comprises a user moving a cursor to the selected menu item and the air-tap gesture input comprises the user tapping the air subsequent to aforesaid cursor movement; and

performing, by the computing device, a function associated with the selected menu item.

14. The method of claim 13 , further comprising:

accepting, by the computing device, a combination of the cursor input and a snap of fingers for selecting the one of the plurality of different selectable menu items.

15. A method for allowing a user to interface with a computing device, comprising:

displaying, with a display device, a multimodal graphical user interface, wherein the multimodal graphical user interface comprise a display of a menu of a plurality of different selectable menu items, wherein the display of each of the plurality of different selectable menu items includes a command name and a unique hand shape corresponding to the command name;

accepting, by the computing device, a combination of a cursor input and another input for selecting one of the plurality of different selectable menu items, wherein the another input comprises an air-tap gesture; and

performing, by the computing device, a function associated with the selected menu item.

16. The method of claim 15 , wherein the another input further includes a hand shape gesture input corresponding to the unique hand shape in the plurality of different selectable menu items.

17. The method of claim 15 , wherein the another input further includes a snap of fingers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 24, 2012
From: MADHVANATH, SRIGANESH; FREEMAN, DUSTIN
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 028627/0739 →
Priority Claims (1)
IN 2684/CHE/2011 · Jul 28, 2011 · national
Continuity (1)
Related Publication 20130030815A1 · Jan 31, 2013