IP Library Granted Patent US 12,099,660
Granted Patent B2
US 12,099,660 · App. 17/984,964 · Granted Sep 24, 2024

User-defined virtual interaction space and manipulation of virtual cameras in the interaction space

Inventors: Isaac Cohen (Oakland, CA); Maxwell Sills (San Francisco, CA)
Assignee: Ultrahaptics IP Two Limited
G06F3/017G06F3/011G06F3/0304G06F3/04815G06F3/04842G06F3/04845G06T15/20G06F2203/04806
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,099,660
App. No.
17/984,964
Granted
Sep 24, 2024
Kind
B2
Abstract

The technology disclosed relates to creating user-defined interaction spaces and modalities in a three dimensional (3D) sensor space in response to control gestures. It also relates to controlling virtual cameras in the 3D sensor space using control gestures and manipulating controls of the virtual cameras through the control gestures. In particular, it relates to defining one or more spatial attributes of the interaction spaces and modalities in response to one or more gesture parameters of the control gesture. It also particularly relates to defining one or more visual parameters of a virtual camera in response to one or more gesture parameters of the control gesture.

Claims (47)

1. A method for defining interface modalities in a three dimensional (3D) sensor space, the method including:

detecting a control gesture made by a control object moving in a three dimensional (3D) sensor space, wherein the control gesture includes at least one portion having a sweep that defines a collection of points within a distance to a point in the 3D sensor space;

obtaining at least one gesture parameter for the control gesture; and defining spatial attributes for an interaction modality in the 3D sensor space responsive to the at least one gesture parameter of the control gesture.

2. The method of claim 1 , wherein the at least one gesture parameter includes at least one of a length and a width of the control gesture.

3. The method of claim 1 , wherein the at least one gesture parameter includes at least one of a structure, a scale, an orientation, or a density of the control object.

4. The method of claim 1 , wherein the spatial attributes include at least one of a height and a width of an interaction space.

5. The method of claim 1 , wherein the spatial attributes include at least numerosity of elements in the interaction modality.

6. The method of claim 1 , further including detecting a context-setting control gesture that identifies a context for interpreting a subsequent control gesture that defines spatial attributes of the interaction modality.

7. The method of claim 6 , wherein the context-setting control gesture is a voice, visual, or device command.

8. The method of claim 6 , wherein the subsequent control gesture applies to an entire interaction space.

9. The method of claim 6 , wherein the subsequent control gesture applies to an element of virtual interaction space.

10. The method of claim 6 , further including creating context-aware elements of the interaction modality that automatically interpret the context-setting control gesture and the subsequent control gesture to define spatial attributes of the interaction modality.

11. The method of claim 1 , wherein the control gesture is a stroke of a user appendage.

12. The method of claim 1 , wherein the control object is a detectable object and the control gesture defines a set of points that have at least one parameter in common within a threshold deviation.

13. The method of claim 1 , wherein a threshold deviation is determined by a variation in angle along velocity vectors over time.

14. The method of claim 1 , the method further including:

defining a vertical extent of a virtual interaction space in proportion to length of vertical portion of the sweep made by the control object;

defining a horizontal extent of the virtual interaction space in proportion to width of horizontal portion of the sweep made by the control object; and

manipulating controls in a physical interaction space by superimposing the virtual interaction space on the physical interaction space responsive to the vertical extent and horizontal extent.

15. The method of claim 1 , the method further including:

defining a vertical extent of a virtual interaction space in proportion to length of vertical portion of the sweep made by the control object;

defining a horizontal extent of the virtual interaction space in proportion to width of horizontal portion of the sweep made by the control object; and

manipulating controls in a synthetic interaction space by linking the virtual interaction space to an image responsive to the vertical extent and horizontal extent.

16. The method of claim 1 , the method further including:

detecting a circular sweep of a control object responsive to a control gesture in a three dimensional (3D) sensor space;

calculating a radius for the circular sweep based on a found point that is equidistant to a plurality of points defined on contour of the control gesture;

constructing a radial-based virtual interaction modality in the 3D sensor space that is in proportion to the radius of the circular sweep; and

manipulating controls in a physical interaction space by superimposing the radial-based virtual interaction modality on the physical interaction space responsive to the circular sweep.

17. The method of claim 1 , the method further including:

identifying a pair of starting points in respective centers of control points of one or more control objects that are detected in a three dimensional (3D) sensor space, wherein the pair of starting points are fixed distance apart;

detecting an outward expanding movement of control points of one or more control objects in the 3D sensor space;

identifying a pair of resting points in respective centers of control points of one or more control objects when control points of one or more control objects come to rest;

defining a horizontal extent of a virtual interaction space in proportion to a distance between starting points and resting points;

defining a vertical extent of the virtual interaction space in proportion to width of one or more control objects; and

presenting the virtual interaction space responsive to the vertical extent and horizontal extent.

18. The method of claim 17 , wherein control objects are hands and control points are finger tips.

19. A non-transitory computer readable medium storing instructs for defining interface modalities in a three dimensional (3D) sensor space, which instructions, when executed by one or more processors perform actions including:

detecting a control gesture made by a control object moving in a three dimensional (3D) sensor space, wherein the control gesture includes at least one portion having a sweep that defines a collection of points within a distance to a point in the 3D sensor space;

obtaining at least one gesture parameter for the control gesture; and

defining spatial attributes for an interaction modality in the 3D sensor space responsive to the at least one gesture parameter of the control gesture.

20. A system, for controlling a machine, including:

at least one camera;

one or more processors coupled to the at least one camera to process image information;

a computer readable medium storing instructions or defining interface modalities in a three dimensional (3D) sensor space, which instructions, when executed by one or more processors perform actions including:

detecting a control gesture made by a control object moving in a three dimensional (3D) sensor space, wherein the control gesture includes at least one portion having a sweep that defines a collection of points within a distance to a point in the 3D sensor space;

obtaining at least one gesture parameter for the control gesture; and

defining spatial attributes for an interaction modality in the 3D sensor space responsive to the at least one gesture parameter of the control gesture.

Assignments (6)
SECURITY INTEREST Recorded Apr 6, 2026
From: SIM IP HXR LLC
To: UNITY MASTER LLC SERIES XIX
Reel/Frame 075365/0907 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2026
From: ULTRAHAPTICS IP TWO LIMITED
To: SIM IP HXR LLC
Reel/Frame 075127/0793 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 16, 2026
From: ULTRAHAPTICS LIMITED; ULTRAHAPTICS IP LIMITED; ULTRAHAPTICS IP TWO LIMITED; ULTRALEAP LIMITED
To: SIM IP HXR LLC
Reel/Frame 074403/0664 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 10, 2023
From: COHEN, ISAAC; SILLS, MAXWELL
To: LEAP MOTION, INC.
Reel/Frame 062661/0883 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 10, 2023
From: LEAP MOTION, INC.
To: LMI LIQUIDATING CO. LLC
Reel/Frame 062661/0952 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 10, 2023
From: LMI LIQUIDATING CO. LLC
To: ULTRAHAPTICS IP TWO LIMITED
Reel/Frame 062662/0031 →