IP Library Granted Patent US 12,333,081
Granted Patent B2
US 12,333,081 · App. 17/409,767 · Granted Jun 17, 2025

Interacting with a machine using gestures in first and second user-specific virtual planes

Inventors: Avinash Dabir (San Francisco, CA); Paul Durdik (Foster City, CA); Keith Mertens (Oakland, CA); Michael Zagorsek (Yountville, CA)
Assignee: Ultrahaptics IP Two Limited
G06F3/017G06V40/11G06V40/28
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,333,081
App. No.
17/409,767
Granted
Jun 17, 2025
Kind
B2
Abstract

Methods and systems for processing an input are disclosed that detect a portion of a hand and/or other detectable object in a region of space monitored by a 3D sensor. The method further includes determining a zone corresponding to the region of space in which the portion of the hand or other detectable object was detected. Also, the method can include determining from the zone a correct way to interpret inputs made by a position, shape or a motion of the portion of the hand or other detectable object.

Claims (35)

1. A method of interacting with a machine using input gestures, the method comprising:

detecting a first finger state of a first finger of a hand of a user relative to a first user-specific virtual plane in a region of space monitored by a 3D sensor, the first finger state being one of the first finger moving closer to or further away from the first user-specific virtual plane;

detecting a second finger state of a second finger of the hand of the user relative to a second user-specific virtual plane in the region of space monitored by the 3D sensor, wherein (i) the second user-specific virtual plane is independent of the first user-specific virtual plane and (ii) the first finger and the second finger belong to the same hand of the user;

interpreting the first finger state as a first input gesture command to interact with a first functionality of the machine, wherein the first input gesture command is relative to the first user-specific virtual plane; and

interpreting the second finger state as a second input gesture command to interact with a second functionality of the machine, wherein the second input gesture command is relative to the second user-specific virtual plane.

2. The method of claim 1 , further comprising interpreting the first input gesture command to be a pinch gesture command to zoom-in.

3. The method of claim 1 , further comprising interpreting the second input gesture command to be a spreading gesture command to zoom-out.

4. The method of claim 1 , further comprising interpreting the first input gesture command to be a pressure gesture command.

5. The method of claim 1 , further comprising determining, from sensed positional information of the first finger or the second finger, a speed of one or more fingers of the user and adjusting gesture-recognition sensitivity based on the determined speed.

6. The method of claim 1 , further comprising:

dynamically personalizing one or more planes by characterizing trajectories of user gestures as vectors in 3-dimensional space, using the vectors to determine a maximal depth from gestural movement along an axis extending from the user to a display; and

defining a thickness of a plane based upon the maximal depth.

7. The method of claim 6 , further comprising revising plane positions as more user movements are detected.

8. The method of claim 1 , further comprising shifting a plane in space as the user changes position.

9. The method of claim 1 , further comprising determining a correct way to interpret an input detected by the 3D sensor including:

using sensed positional information of the first finger, determining, from a plurality of zones defined for the region, a zone in which the first finger is present at a time the first finger state is detected;

determining an active program and that the determined zone is a hover zone for the active program; and

interpreting a position or a motion of the first finger based on the active program and the hover zone.

10. The method of claim 1 , further comprising:

interpreting a motion of the first finger as one or more gestures; and

determining, from the one or more gestures, an input to at least one of (i) an application and (ii) an operating system.

11. A system for interacting with a machine using input gestures, the system including an image analyzer comprising a processor configured to perform the method of claim 1 .

12. A non-transitory computer-readable storage medium storing instructions for interacting with a machine using input gestures, the instructions, when executed by a processor, cause the processor to perform the method of claim 1 .

13. The method of claim 1 , wherein the first user-specific virtual plane is defined based on the first finger and the second user-specific virtual plane is defined based on the second finger.

14. The method of claim 1 , wherein a location and an orientation of at least one user-specific virtual plane are defined in the region of space independently from one of:

a location and an orientation of another body part of the user, and

a location and an orientation of the machine.

15. A method of interacting with a machine using input gestures, the method comprising:

detecting a first finger state of a first finger of a hand of a user relative to a first user-specific virtual plane in a region of space monitored by a 3D sensor, the first finger state being one of the first finger moving on or against the first user-specific virtual plane;

detecting a second finger state of a second finger of the hand of the user relative to a second user-specific virtual plane in the region of space monitored by the 3D sensor, wherein the first finger and the second finger belong to the same hand of the user;

interpreting the first finger state as a first input gesture command to interact with a first functionality of the machine, wherein the first input gesture command is (i) relative to the first user-specific virtual plane and (ii) independent of the second-user specific virtual plane; and

interpreting the second finger state as a second input gesture command to interact with a second functionality of the machine, wherein the second input gesture command is (i) distinct from the first input gesture command, (ii) relative to the second user-specific virtual plane, and (iii) independent of the first user-specific virtual plane.

16. A system for interacting with a machine using input gestures, the system including an image analyzer including a processor configured to perform the method of claim 15 .

17. A non-transitory computer-readable storage medium storing instructions for interacting with a machine using input gestures, the instructions, when executed by a processor, cause the processor to perform the method of claim 15 .

18. The method of claim 15 , wherein the interpreting of the second finger state as a second input gesture is performed during the interpreting of the first finger state as a first input gesture.

Assignments (5)
SECURITY INTEREST Recorded Apr 6, 2026
From: SIM IP HXR LLC
To: UNITY MASTER LLC SERIES XIX
Reel/Frame 075365/0907 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2026
From: ULTRAHAPTICS IP TWO LIMITED
To: SIM IP HXR LLC
Reel/Frame 075127/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2021
From: ZAGORSEK, MICHAEL; DABIR, AVINASH; DURDIK, PAUL; MERTENS, KEITH
To: LEAP MOTION, INC.
Reel/Frame 058012/0032 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2021
From: LEAP MOTION, INC.
To: LMI LIQUIDATING CO. LLC
Reel/Frame 058012/0059 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2021
From: LMI LIQUIDATING CO. LLC
To: ULTRAHAPTICS IP TWO LIMITED
Reel/Frame 058012/0082 →