Gesture management system
View Patent ↗For storing gesture definitions and evaluating expressions that reference the gesture definitions, an expression evaluation engine evaluates the expressions to determine whether movements of a user satisfy the expressions. The expression evaluation engine receives expressions in user or application requests, or the expression evaluation engine may automatically evaluate the expressions when a gesture recognition system receives updated information about tracked body parts of the user. An expression includes a logical operator that relates different conditions to each other, and the different conditions may reference different gestures or other criteria that are based on the tracked body parts of the user. The expression is satisfied when the logical operator is satisfied. If an expression is satisfied, the expression evaluation engine triggers an action associated with a corresponding gesture definition. An example gesture may trigger display and navigation of a pie menu.
1. A method comprising:
receiving a first input defining a first gesture definition as a condition regarding a first tracked body part;
storing the first gesture definition in the gesture dictionary;
receiving a second input defining a second gesture definition as a condition regarding a second tracked body part;
storing the second gesture definition in the gesture dictionary;
receiving a third input defining a third gesture definition as a condition regarding a third tracked body part;
storing the third gesture definition in the gesture dictionary;
receiving a fourth input defining a fourth gesture definition as a condition regarding a fourth tracked body part;
storing the fourth gesture definition in the gesture dictionary;
receiving, as definition input, a first reference to a first gesture definition that is defined in a gesture dictionary, a second reference to a second gesture definition defined in the gesture dictionary, and a first operator;
wherein the first input defines the first gesture definition as a third reference to the third gesture definition that is defined in the gesture dictionary, a fourth reference to the fourth gesture definition defined in the gesture dictionary, and a second operator;
based on said definition input, storing a complex gesture definition as a combination of the first gesture definition, the second gesture definition, and the first operator;
receiving, in an application program that is controlled using gestures as a form of user input, an action to execute in the application program;
causing the action to be triggered in said application program in response to user input satisfying the complex gesture definition;
wherein the method is performed on one or more computing devices.
2. The method of claim 1 , wherein the first operator is selected from a group consisting of a sequence operators and Boolean operators.
3. The method of claim 1 , wherein the first operator is a first occurrence of a particular sequence operator, and the second operator is a second occurrence of the same particular sequence operator causing the action to be triggered in said application program only in response to user input satisfying the third gesture definition, the fourth gesture definition, and then the second gesture definition in that sequence.
4. The method of claim 1 , wherein said application program is implemented in conjunction with skeletal tracking software and hardware that processes user input as a plurality of tracked body parts.
5. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause performance of:
receiving a first input defining a first gesture definition as a condition regarding a first tracked body part;
storing the first gesture definition in the gesture dictionary;
receiving a second input defining a second gesture definition as a condition regarding a second tracked body part;
storing the second gesture definition in the gesture dictionary;
receiving a third input defining a third gesture definition as a condition regarding a third tracked body part;
storing the third gesture definition in the gesture dictionary;
receiving a fourth input defining a fourth gesture definition as a condition regarding a fourth tracked body part;
storing the fourth gesture definition in the gesture dictionary;
receiving, as definition input, a first reference to a first gesture definition that is defined in a gesture dictionary, a second reference to a second gesture definition defined in the gesture dictionary, and a first operator;
wherein the first input defines the first gesture definition as a third reference to the third gesture definition that is defined in the gesture dictionary, a fourth reference to the fourth gesture definition defined in the gesture dictionary, and a second operator;
based on said definition input, storing a complex gesture definition as a combination of the first gesture definition, the second gesture definition, and the first operator;
receiving, in an application program that is controlled using gestures as a form of user input, an action to execute in the application program; and
causing the action to be triggered in said application program in response to user input satisfying the complex gesture definition.
6. The one or more non-transitory computer-readable media of claim 5 , wherein the first operator is selected from a group consisting of a sequence operators and Boolean operators.
7. The one or more non-transitory computer-readable media of claim 5 , wherein the first operator is a first occurrence of a particular sequence operator, and the second operator is a second occurrence of the same particular sequence operator causing the action to be triggered in said application program only in response to user input satisfying the third gesture definition, the fourth gesture definition, and then the second gesture definition in that sequence.
8. The one or more non-transitory computer-readable media of claim 5 , wherein said application program is implemented in conjunction with skeletal tracking software and hardware that processes user input as a plurality of tracked body parts.
9. A gesture management system, comprising:
one or more hardware processors;
one or more non-transitory computer-readable data storage media storing sequences of instructions which, when executed using the one or more hardware processors, cause the one or more hardware processors to:
receiving a first input defining a first gesture definition as a condition regarding a first tracked body part;
storing the first gesture definition in the gesture dictionary;
receiving a second input defining a second gesture definition as a condition regarding a second tracked body part;
storing the second gesture definition in the gesture dictionary;
receiving a third input defining a third gesture definition as a condition regarding a third tracked body part;
storing the third gesture definition in the gesture dictionary;
receiving a fourth input defining a fourth gesture definition as a condition regarding a fourth tracked body part;
storing the fourth gesture definition in the gesture dictionary;
receive, as definition input, a first reference to a first gesture definition that is defined in a gesture dictionary, a second reference to a second gesture definition defined in the gesture dictionary, and a first operator;
wherein the first input defines the first gesture definition as a third reference to the third gesture definition that is defined in the gesture dictionary, a fourth reference to the fourth gesture definition defined in the gesture dictionary, and a second operator;
based on said definition input, store a complex gesture definition as a combination of the first gesture definition, the second gesture definition, and the first operator;
receive, in an application program that is controlled using gestures as a form of user input, an action to execute in the application program; and
cause the action to be triggered in said application program in response to user input satisfying the complex gesture definition.
10. The gesture management system of claim 9 , wherein the first operator is selected from a group consisting of a sequence operators and Boolean operators.
11. The gesture management system of claim 9 , wherein the first operator is a first occurrence of a particular sequence operator, and the second operator is a second occurrence of the same particular sequence operator causing the action to be triggered in said application program only in response to user input satisfying the third gesture definition, the fourth gesture definition, and then the second gesture definition in that sequence.