Method for creating a gesture library
The technology disclosed relates to the creation of a gesture library for subsequent use in filtering gestures. In particular, the methods disclosed here generate and store gestures and their characteristic values to create a set of user-defined reference gestures that can be compared against characteristics of actual gestures performed in a 3D sensory space. Based on these comparisons, a set of gestures of interest may be filtered from all the gestures performed in the 3D sensory space. The technology disclosed also relates to customizing gesture interpretation for a particular user, and to setting parameters for recognizing gestures by prompting the user to select values for characteristics of the gestures. In one implementation, the technology disclosed includes performing characteristic focused demonstrations of boundaries of the gesture. It further includes testing the interpretation of gestures by prompting the user to perform complete gesture demonstrations and receiving user evaluation regarding the interpretation.
1 . A method including:
prompting a demonstration of a first gesture in a three-dimensional (3D) space, the first gesture being associated with a first action;
recognizing a first performance of the first gesture associated with the first action;
recognizing a second performance of the first gesture associated with the first action;
creating a first data object for the first gesture and associating the first data object with the first action;
recognizing a performance of a second gesture;
determining that the recognized second gesture is associated with the first action based, at least in part, on comparing the recognized second gesture to the first data object; and
executing the first action.
2 . The method of claim 1 , wherein
creating the first data object for the first gesture is based, at least in part, on the first performance and the second performance of the first gesture associated with the first action.
3 . The method of claim 1 , wherein
recognizing at least one of the first or the second performance of the first gesture is executed based, at least in part, on a motion-capture (mocap) program.
4 . The method of claim 1 , wherein
the first data object comprises a first set of parameters.
5 . The method of claim 1 , wherein
the first data object comprises at least a first parameter range.
6 . The method of claim 1 ,
wherein the first data object comprises a first set of parameters and a first parameter range;
wherein comparing the recognized second gesture to the first data object comprises determining a second set of parameters for the second gesture; and
wherein determining the recognized second gesture is associated with the first action comprises determining that at least one parameter of the second set of parameters is within the first parameter range.
7 . The method of claim 1 ,
wherein the first data object comprises a first set of parameters;
wherein comparing the recognized second gesture to the first data object comprises determining a second set of parameters for the second gesture; and
wherein the method includes modifying the first data object based, at least in part, on the determined second set of parameters for the second gesture.
8 . The method of claim 1 ,
wherein the first data object comprises a first set of parameters and a first parameter range;
wherein comparing the recognized second gesture to the first data object comprises determining a second set of parameters for the second gesture; and
wherein the method includes modifying the first parameter range based, at least in part, on the second set of parameters for the second gesture.
9 . The method of claim 1 , wherein
prompting the demonstration of a first gesture in a three-dimensional (3D) space comprises providing one or more prompts requesting performance of a demonstration of boundaries of the first gesture.
10 . The method of claim 1 ,
wherein prompting the demonstration of a first gesture in a three-dimensional (3D) space comprises providing one or more questions requesting one or more responses about characteristics of the first gesture.
11 . The method of claim 1 , wherein
prompting the demonstration of a first gesture in a three-dimensional (3D) space comprises providing one or more questions requesting one or more responses about characteristics of the first gesture, the questions including at least one of:
a question regarding a minimum and maximum amount of time the first gesture takes to perform,
a question regarding a threshold time-period,
a question regarding a number of hands and fingers involved in making the first gesture, or
a question regarding whether size, speed, or direction of the first gesture matters.
12 . The method of claim 1 , comprising:
recognizing a performance of a third gesture;
comparing the recognized third gesture to the first data object; and
determining the recognized third gesture is associated with a second action.
13 . The method of claim 12 ,
wherein the first data object comprises a first set of parameters and a first parameter range; and
wherein comparing the recognized third gesture to the first data object comprises determining a third set of parameters for the third gesture.
14 . The method of claim 12 ,
wherein the first data object comprises a first set of parameters and a first parameter range;
wherein comparing the recognized third gesture to the first data object comprises determining a third set of parameters of for the third gesture;
wherein a second data object comprises a second parameter range; and
wherein the method comprises:
determining that at least one parameter of the third set of parameters is outside the first parameter range of the first data object;
determining that at least one parameter of the third set of parameters is within the second parameter range of the second data object; and
executing the second action.
15 . The method of claim 12 , comprising:
storing data associating the third gesture with the second action, the second action being different from the first action.
16 . The method of claim 1 , wherein the first data object comprises a first set of parameters that include at least one of:
a path of a gesture, parameters of a gesture, spatial properties of the gesture, temporal properties of the gesture, or a velocity of the gesture.
17 . The method of claim 1 , comprising:
storing data associating the first gesture and the second gesture with the first action.
18 . A system including one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement operations comprising:
prompting a demonstration of a first gesture in a three-dimensional (3D) space, the first gesture being associated with a first action;
recognizing a first performance of the first gesture associated with the first action;
recognizing a second performance of the first gesture associated with the first action;
creating a first data object for the first gesture and associating the first data object with the first action;
recognizing a performance of a second gesture;
determining that the recognized second gesture is associated with the first action based, at least in part, on comparing the recognized second gesture to the first data object; and
executing the first action.
19 . A non-transitory computer-readable storage medium storing computer program instructions that, when executed on a processor, implement operations including:
prompting a demonstration of a first gesture in a three-dimensional (3D) space, the first gesture being associated with a first action;
recognizing a first performance of the first gesture associated with the first action;
recognizing a second performance of the first gesture associated with the first action;
creating a first data object for the first gesture and associating the first data object with the first action;
recognizing a performance of a second gesture;
determining that the recognized second gesture is associated with the first action based, at least in part, on comparing the recognized second gesture to the first data object; and
executing the first action.