Granted Patent B2

US 12,204,695 · App. 18/219,517 · Granted Jan 21, 2025

Inventors: Raffi Bedikian (San Francisco, CA); Jonathan Marsden (San Mateo, CA); Keith Mertens (Oakland, CA); David Holz (San Francisco, CA); Maxwell Sills (San Francisco, CA); Matias Perez (San Francisco, CA); Gabriel Hare (Daly City, CA); Ryan Julian (Berkeley, CA)

Assignee: Ultrahaptics IP Two Limited

G06F3/017G06F3/0325G06F3/04815G06V40/20

Abstract

Claims (60)

1. A method of controlling a machine, comprising:

sensing a variation of position of a control object using an imaging system;

determining, from the variation, one or more primitives describing a characteristic of a control object moving in space, and determining whether motion of the control object corresponds to an engagement gesture;

comparing the one or more primitives to one or more engagement gesture templates in a library of gesture templates;

selecting, based on a result of the comparing, one or more engagement gesture templates corresponding to the one or more primitives;

providing, responsive to the variation, at least one engagement gesture template of the selected one or more engagement gesture templates as an indication of a command to issue to a machine under control;

computationally determining a degree of completion of at least one gesture; and

modifying contents of a display in accordance with the engagement gesture.

2. The method according to claim 1 , further comprising:

computationally determining a degree of completion of at least one gesture; and

wherein modifying contents of a display in accordance with the engagement gesture also comprises modifying contents of the display in accordance of the determined degree of completion, such that the display gradually fills in an indicator with color or shading indicating how close motion of the control object is to completing the gesture.

3. The method according to claim 2 , further comprising:

identifying a scale associated with the control object, the scale being indicative of an actual distance traversed by the control object; and

computationally determining a ratio between the scale and a displayed movement corresponding to an action to be displayed on a presentation device;

displaying the action on the presentation device based on the ratio; and

adjusting the ratio based on an external parameter.

4. The method according to claim 3 , wherein

the external parameter is a ratio of a pixel distance in captured images to a size, in pixels, of a display screen of the presentation device.

5. The method according to claim 1 , wherein the comparing comprises:

disassembling at least a portion of a trajectory of the control object into a set of frequency components and

searching for a set of frequency components among the one or more gesture templates.

6. The method according to claim 1 , wherein the comparing comprises:

disassembling at least a portion of a trajectory of the control object into a set of time dependent frequency components; and

searching for a set of time dependent frequency components among the one or more gesture templates stored in the library.

7. The method according to claim 6 , wherein the disassembling comprises applying wavelet analysis to the portion of the trajectory as a signal over time to determine the set of time dependent frequency components.

8. The method according to claim 1 , wherein the selecting comprises:

determining a similarity between the one or more primitives and the one or more gesture templates by applying at least one similarity determiner; and

providing the similarity as an indication of a quality of correspondence between the one or more primitives and the one or more gesture templates.

9. The method according to claim 1 , wherein the selecting comprises:

performing at least one of scaling and shifting to at least one of the one or more primitives and the one or more gesture templates.

10. The method according to claim 1 , wherein the selecting comprises:

disassembling at least a portion of a trajectory of the control object into a set of frequency components;

filtering the set of frequency components to remove motions associated with jitter; and

searching for a gesture template, of the one or more gesture templates, that matches a frequency component of the filtered set of frequency components.

11. The method according to claim 10 , wherein the filtering comprises applying a Frenet-Serret filter.

12. The method according to claim 1 , wherein determining comprises determining a position or motion of the control object relative to a virtual control construct.

13. The method according to claim 1 , further comprising:

identifying two simultaneous gestures based on variations of positions of the control object;

computationally determining a dominant gesture of the two simultaneous gestures; and

presenting an action on a presentation device based on the dominant gesture.

14. The method according to claim 1 , wherein the providing comprises:

detecting a conflict between a gesture template corresponding to a user-defined gesture and a gesture template corresponding to a predetermined gesture; and

applying a resolution determiner to resolve the conflict by selecting one of: the user-defined gesture and the conflicting predetermined gesture.

15. The method according to claim 14 , wherein the applying comprises selecting the user-defined gesture when the conflict is between the predetermined gesture and the user-defined gesture.

16. A system comprising a memory and one or more processors, the memory being loaded with computer instruction that, when executed on the one or more processors, cause the one or more processors to implement operations comprising:

sensing a variation of position of a control object using an imaging system;

determining, from the variation, one or more primitives describing a characteristic of a control object moving in space, and whether motion of the control object corresponds to an engagement gesture;

comparing the one or more primitives to one or more engagement gesture templates in a library of gesture templates;

selecting, based on a result of the comparing, one or more engagement gesture templates corresponding to the one or more primitives;

providing, responsive to the variation, at least one engagement gesture template of the selected one or more engagement gesture templates as an indication of a command to issue to a machine under control;

computationally determining a degree of completion of at least one gesture; and

modifying contents of a display in accordance with the engagement gesture.

17. A non-transitory computer readable recording medium having computer instructions recorded thereon, the computer instructions, when executed by one or more processor, causing the one or more processor to implement operations comprising:

sensing a variation of position of a control object using an imaging system;

determining, from the variation, one or more primitives describing a characteristic of a control object moving in space, and whether motion of the control object corresponds to an engagement gesture;

comparing the one or more primitives to one or more engagement gesture templates in a library of gesture templates;

selecting, based on a result of the comparing, one or more engagement gesture templates corresponding to the one or more primitives;

providing, responsive to the variation, at least one engagement gesture template of the selected one or more engagement gesture templates as an indication of a command to issue to a machine under control;

computationally determining a degree of completion of at least one gesture; and

modifying contents of a display in accordance with the engagement gesture.

Assignments (6)

Continuity (19)

References Cited (400)