GESTURE-LEVEL GRAMMAR ARRANGEMENTS FOR SPATIAL-GESTURE USER INTERFACES SUCH AS TOUCHSCREEN, HIGH-DIMENSIONAL TOUCH PAD (HDTP), FREE-SPACE CAMERA, AND OTHER USER INTERFACE TECHNOLOGIES
A system for a spatial-gesture user interface employing grammatical rules at gesture levels. Real-time spatial-gesture information measured by a spatial-gesture user interface is processed to at least a recognized sequence of specific gestures and that the sequence of gestures that the user's execution of a gesture sequence has been completed. Many additional features are then provided from this foundation, including gesture grammars, structured-meaning gesture-lexicon, imposed interpretations, context, and gesture-rendering prosody. The invention can be used to provide very general spatial-gesture grammar user interface for touchscreens, high dimensional touch pad (HDTP), free-space camera, and other user interfaces. In an implementation, each individual gesture rendered by the user can be recognized according to the sequence of recognized gestemes derived from information measured by a spatial-gesture user interface. In an implementation, various distinct subsets of gestemes can be concatenated in space and time to construct a sequence of individual gestures.
1 . A system implementing a spatial-gesture user interface, the system comprising:
at least one processor for running at least one algorithm, the least one algorithm for receiving an input stream of spatial measurements made over an interval of time by at least one spatial user interface sensor, the algorithm further configured to
recognize each of a sequence of individual gestures from the input stream of spatial measurements, and
impose an interpretation on the sequence of gestures using at least a first grammatical rule,
wherein the algorithm reports at least the interpretation as output information for controlling an aspect of a software application.
2 . The system of claim 1 , wherein prosody is used as an input into the first grammatical rule.
3 . The system of claim 1 , wherein context is determined at least in part from prosody.
4 . The system of claim 1 , wherein a context is determined at least in part from at least one gesture in the sequence of individual gestures.
5 . The system of claim 1 , wherein a context is determined at least in part from at least one earlier-provided gesture.
6 . The system of claim 1 , wherein the first grammatical rule comprises a structured-meaning gesture-lexicon framework for interpreting the sequence of gestures.
7 . The system of claim 1 , wherein at least a second grammatical rule is used in the recognition of at least one of the gestures from the sequence of gestures.
8 . The system of claim 7 , wherein the second grammatical rule comprises a structured-meaning gesture-lexicon framework for interpreting the sequence of gestures.
9 . The system of claim 7 , wherein prosody is used as an input into the second grammatical rule.
10 . A system for controlling an application operating on a processor with a spatial-gesture user interface, the system comprising:
a processor for
receiving a gesture information measured by a spatial-gesture user interface and for executing an algorithm for recognizing gestures comprising a plurality of primitive gesture segments,
recognizing individual gestures from sequences of gestemes, and from this transforming the sequences of gestemes into a sequence of gestures, and
imposing an interpretation of the first specific gesture and second specific gesture based on at least a first grammatical rule,
wherein the algorithm transforms at least two of (a) the sequence of gestures, (b) the imposed interpretation, and (c) a provided context into a particular command for use in controlling a software application; and
wherein the system transmits the particular command to the application.
11 . The system of claim 1 , wherein prosody is used as an input into the first grammatical rule.
12 . The system of claim 1 , wherein context is determined at least in part from prosody.
13 . The system of claim 1 , wherein a context is determined at least in part from at least one gesture in the sequence of individual gestures.
14 . The system of claim 1 , wherein a context is determined at least in part from at least one earlier-provided gesture.
15 . The system of claim 1 , wherein the first grammatical rule comprises a structured-meaning gesture-lexicon framework for interpreting the sequence of gestures.
16 . The system of claim 1 , wherein at least a second grammatical rule is used in the recognition of at least one of the first specific gestures and the second specific gesture from the sequence of gestures.
17 . The system of claim 16 , wherein the second grammatical rule comprises a structured-meaning gesture-lexicon framework for interpreting the sequence of gestures.
18 . The system of claim 16 , wherein prosody is used as an input into the second grammatical rule.