Granted Patent B2

US 12,436,622 · App. 18/883,875 · Granted Oct 7, 2025

Inventors: Isaac Cohen (Oakland, CA); David S. Holz (San Francisco, CA); Maxwell Sills (San Francisco, CA)

Assignee: Ultrahaptics IP Two Limited

G06F3/017G06F3/04815G06F3/0483G06F3/04842G06F3/04847G06F3/0485G06V40/28

Abstract

Claims (48)

1. A method of manipulating one or more objects depicted in a display based on interpreting a movement of a control object, the method including:

sensing a movement of the control object through a 3D sensor space using a 3D sensor;

interpreting, by a computing device, the movement of the control object through the 3D sensor space and comparing the movement of the control object to a control plane defined as tangential to a surface of the control object;

determining by a computing device whether the control object engages a virtual control based on a location of the control object;

sensing a movement of a sub-object of the control object through the 3D sensor space using the 3D sensor;

processing the interpreted movement of the sub-object of the control object; and

interpreting by the computing device a direction of the movement of the sub-object as a gesture controlling the virtual control.

2. The method of claim 1 , wherein

the sub-object is a portion of the control object.

3. The method of claim 1 , wherein

the control object is a hand and the sub-object is a finger of the hand.

4. The method of claim 1 , wherein:

the control object is a hand and the sub-object of the control object are one of:

a finger, a palm, or a back of the hand.

5. The method of claim 1 , wherein

the virtual control is pre-selected before the movement is sensed.

6. The method of claim 1 , wherein

the movement is interpreted and presented as a path on the display when a speed of the movement exceeds a pre-determined threshold.

7. The method of claim 6 , wherein

the pre-determined threshold is 20 cm per second.

8. The method of claim 6 , wherein

the pre-determined threshold is 30 cm per second.

9. The method of claim 6 , wherein

the pre-determined threshold is 40 cm per second.

10. The method of claim 1 , wherein

the control object is a hand, and the movement is a gesture of the hand.

11. The method of claim 1 , wherein

the 3D sensor comprises one or more cameras that capture images of the 3D sensor space.

12. The method of claim 11 , wherein

the 3D sensor space is illuminated by one or more infrared LEDs, and the cameras are sensitive to infrared light.

13. The method of claim 1 , wherein

the 3D sensor space is provided with ultrasound waves from sonic sources.

14. The method of claim 13 , wherein

the 3D sensor comprises one or more sonic sensors that capture sonic shadows or deflections in the 3D sensor space of the ultrasound waves.

15. A non-transitory computer-readable recording medium having computer instructions recorded thereon, the computer instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising:

sensing a movement of the control object through a 3D sensor space using a 3D sensor;

interpreting, by a computing device, the movement of the control object through the 3D sensor space and comparing the movement of the control object to a control plane defined as tangential to a surface of the control object;

determining by a computing device whether the control object engages a virtual control based on a location of the control object;

sensing a movement of a sub-object of the control object through the 3D sensor space using the 3D sensor;

processing the interpreted movement of the sub-object of the control object; and

interpreting by the computing device a direction of the movement of the sub-object as a gesture controlling the virtual control.

16. A system comprising a memory storing computer instructions and one or more processors, the computer instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

sensing a movement of the control object through a 3D sensor space using a 3D sensor;

interpreting, by a computing device, the movement of the control object through the 3D sensor space and comparing the movement of the control object to a control plane defined as tangential to a surface of the control object;

determining by a computing device whether the control object engages a virtual control based on a location of the control object;

sensing a movement of a sub-object of the control object through the 3D sensor space using the 3D sensor;

processing the interpreted movement of the sub-object of the control object; and

interpreting by the computing device a direction of the movement of the sub-object as a gesture controlling the virtual control.

Assignments (6)

Continuity (8)

References Cited (195)