Granted Patent B2

US 12,333,081 · App. 17/409,767 · Granted Jun 17, 2025

Inventors: Avinash Dabir (San Francisco, CA); Paul Durdik (Foster City, CA); Keith Mertens (Oakland, CA); Michael Zagorsek (Yountville, CA)

Assignee: Ultrahaptics IP Two Limited

G06F3/017G06V40/11G06V40/28

Abstract

Claims (35)

1. A method of interacting with a machine using input gestures, the method comprising:

detecting a first finger state of a first finger of a hand of a user relative to a first user-specific virtual plane in a region of space monitored by a 3D sensor, the first finger state being one of the first finger moving closer to or further away from the first user-specific virtual plane;

detecting a second finger state of a second finger of the hand of the user relative to a second user-specific virtual plane in the region of space monitored by the 3D sensor, wherein (i) the second user-specific virtual plane is independent of the first user-specific virtual plane and (ii) the first finger and the second finger belong to the same hand of the user;

interpreting the first finger state as a first input gesture command to interact with a first functionality of the machine, wherein the first input gesture command is relative to the first user-specific virtual plane; and

interpreting the second finger state as a second input gesture command to interact with a second functionality of the machine, wherein the second input gesture command is relative to the second user-specific virtual plane.

2. The method of claim 1 , further comprising interpreting the first input gesture command to be a pinch gesture command to zoom-in.

3. The method of claim 1 , further comprising interpreting the second input gesture command to be a spreading gesture command to zoom-out.

4. The method of claim 1 , further comprising interpreting the first input gesture command to be a pressure gesture command.

5. The method of claim 1 , further comprising determining, from sensed positional information of the first finger or the second finger, a speed of one or more fingers of the user and adjusting gesture-recognition sensitivity based on the determined speed.

6. The method of claim 1 , further comprising:

dynamically personalizing one or more planes by characterizing trajectories of user gestures as vectors in 3-dimensional space, using the vectors to determine a maximal depth from gestural movement along an axis extending from the user to a display; and

defining a thickness of a plane based upon the maximal depth.

7. The method of claim 6 , further comprising revising plane positions as more user movements are detected.

8. The method of claim 1 , further comprising shifting a plane in space as the user changes position.

9. The method of claim 1 , further comprising determining a correct way to interpret an input detected by the 3D sensor including:

using sensed positional information of the first finger, determining, from a plurality of zones defined for the region, a zone in which the first finger is present at a time the first finger state is detected;

determining an active program and that the determined zone is a hover zone for the active program; and

interpreting a position or a motion of the first finger based on the active program and the hover zone.

10. The method of claim 1 , further comprising:

interpreting a motion of the first finger as one or more gestures; and

determining, from the one or more gestures, an input to at least one of (i) an application and (ii) an operating system.

11. A system for interacting with a machine using input gestures, the system including an image analyzer comprising a processor configured to perform the method of claim 1 .

12. A non-transitory computer-readable storage medium storing instructions for interacting with a machine using input gestures, the instructions, when executed by a processor, cause the processor to perform the method of claim 1 .

13. The method of claim 1 , wherein the first user-specific virtual plane is defined based on the first finger and the second user-specific virtual plane is defined based on the second finger.

14. The method of claim 1 , wherein a location and an orientation of at least one user-specific virtual plane are defined in the region of space independently from one of:

a location and an orientation of another body part of the user, and

a location and an orientation of the machine.

15. A method of interacting with a machine using input gestures, the method comprising:

detecting a first finger state of a first finger of a hand of a user relative to a first user-specific virtual plane in a region of space monitored by a 3D sensor, the first finger state being one of the first finger moving on or against the first user-specific virtual plane;

detecting a second finger state of a second finger of the hand of the user relative to a second user-specific virtual plane in the region of space monitored by the 3D sensor, wherein the first finger and the second finger belong to the same hand of the user;

interpreting the first finger state as a first input gesture command to interact with a first functionality of the machine, wherein the first input gesture command is (i) relative to the first user-specific virtual plane and (ii) independent of the second-user specific virtual plane; and

interpreting the second finger state as a second input gesture command to interact with a second functionality of the machine, wherein the second input gesture command is (i) distinct from the first input gesture command, (ii) relative to the second user-specific virtual plane, and (iii) independent of the first user-specific virtual plane.

16. A system for interacting with a machine using input gestures, the system including an image analyzer including a processor configured to perform the method of claim 15 .

17. A non-transitory computer-readable storage medium storing instructions for interacting with a machine using input gestures, the instructions, when executed by a processor, cause the processor to perform the method of claim 15 .

18. The method of claim 15 , wherein the interpreting of the second finger state as a second input gesture is performed during the interpreting of the first finger state as a first input gesture.

Assignments (6)

Continuity (5)

References Cited (400)