IP Library Granted Patent US 11,994,377
Granted Patent B2
US 11,994,377 · App. 17/010,531 · Granted May 28, 2024

Systems and methods of locating a control object appendage in three dimensional (3D) space

Inventor: David S. Holz (San Francisco, CA)
Assignee: Ultrahaptics IP Two Limited
G01B11/2433G06T7/292G06T7/507G06T7/586G06T7/593G06T17/00G06V10/145G06V20/40G06V20/64G06V40/107G06V40/28G06F3/017G06T17/10G06T2207/10152G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,994,377
App. No.
17/010,531
Granted
May 28, 2024
Kind
B2
Abstract

Methods and systems for capturing motion and/or determining the shapes and positions of one or more objects in 3D space utilize cross-sections thereof. In various embodiments, images of the cross-sections are captured using a camera based on reflections therefrom or shadows cast thereby.

Claims (69)

1. A system for recognizing gestures from a control object moving in three dimensional (3D) space, the system including:

one or more processors coupled to a memory, the memory loaded with computer instructions that, when executed by the one or more processors, implement actions including:

receiving a result of analyzing a gesture;

interpreting the result of analyzing a gesture as an instruction to perform an operation,

wherein the result of analyzing a gesture is obtained from another program, which is executed on a processor, to analyze differences in positions of fit closed curves fit to observed edges of a control object thereby tracking motion of the control object while making a gesture in a 3D space by repeatedly:

obtaining captured images of a control object moving in 3D space captured from at least two geometrically distinct predetermined vantages;

calculating observed edges of the control object from the captured images;

fitting closed curves to the observed edges of the control object as captured in the captured images by selecting a closed curve from a family of similar closed curves that fit the observed edges of the control object as captured using an assumed parameter;

whenever the control object is a complex control object that includes a palm and multiple fingers, applying actions including obtaining captured images, calculating observed edges, and fitting closed curves to construct multiple fingers of control object appendages; and

fitting cross sections of a palm to observed edges of a palm as captured in the captured images.

2. The system of claim 1 , wherein one or more fitted parameters of a first fitted closed curve are substituted for one or more of the observed edges and the assumed parameter when fitting an adjacent second closed curve.

3. The system of claim 1 , wherein a first fitted closed curve is used to filter fits of additional closed curves.

4. The system of claim 1 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

repeatedly applying actions including obtaining captured images, calculating observed edges, and fitting closed curves actions of claim 1 over time; and

calculating motion of the control object over time based on differences between modeled locations of the control object.

5. The system of claim 3 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

repeatedly applying actions including obtaining captured images, calculating observed edges, fitting closed curves and using a first fitted closed curve to filter fits of additional closed curves actions of claim 3 over time; and

calculating motion of a complex control object over time based on differences between modeled locations of a complex control object.

6. The system of claim 1 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

determining and fitting a circle selected from among closed curves for a plurality of portions of the control object from the captured images, including:

calculating three co planar tangents to observed edges of the control object from the captured images; and

fitting a circle to the control object using at least the three co planar tangents.

7. The system of claim 6 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

for a complex control object model that includes a palm and multiple fingers, applying the determining and fitting a circle actions of claim 6 to construct multiple fingers of control object appendages; and

fitting cross sections of a palm to observed edges of the palm as captured in the captured images.

8. The system of claim 6 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

repeatedly applying the determining and fitting a circle actions of claim 6 over time; and

calculating motion of the control object over time based on differences between modeled locations of the control object.

9. The system of claim 7 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

repeatedly applying the determining and fitting a circle and fitting cross sections actions of claim 7 over time; and

calculating motion of a complex control object over time based on differences between modeled locations of the complex control object.

10. A non-transitory computer readable medium storing a plurality of instructions for programming one or more processors, which instructions, when executed by one or more processors, implement actions including:

receiving a result of analyzing a gesture;

interpreting as an instruction the result of analyzing a gesture; wherein the result of analyzing a gesture is obtained from another program executing on a processor from analysis of differences in positions of fit closed curves fit to observed edges of a control object thereby tracking motion of the control object while making a gesture in a 3D space including repeatedly:

obtaining captured images of a control object moving in 3D space captured from at least two geometrically distinct predetermined vantages;

calculating observed edges of the control object from the captured images;

fitting closed curves to the observed edges of the control object as captured in the captured images by selecting a closed curve from a family of similar closed curves that fit the observed edges of the control object as captured using an assumed parameter; and

whenever the control object is a complex control object that includes a palm and multiple fingers, applying actions including obtaining captured images, calculating observed edges, and fitting closed curves actions to construct multiple fingers of control object appendages; and

fitting cross sections of a palm to observed edges of a palm as captured in the captured images.

11. The non-transitory computer readable medium of claim 10 ,

wherein one or more fitted parameters of a first fitted closed curve are substituted for one or more of the observed edges and the assumed parameter when fitting an adjacent second closed curve.

12. The non-transitory computer readable medium of claim 10 , wherein a first fitted closed curve is used to filter fits of additional closed curves to contiguous cross-sections.

13. The non-transitory computer readable medium of claim 10 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

repeatedly applying actions including obtaining captured images, calculating observed edges, and fitting closed curves actions of claim 10 overtime; and

calculating motion of the control object over time based on differences between modeled locations of the control object.

14. The non-transitory computer readable medium of claim 12 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

repeatedly applying actions including obtaining captured images, calculating observed edges, fitting closed curves and using a first fitted closed curve to filter fits of additional closed curves actions of claim 12 overtime; and

calculating motion of a complex control object over time based on differences between modeled locations of a complex control object.

15. The non-transitory computer readable medium of claim 10 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

determining and fitting a circle selected from among closed curves in the family of similar closed curves for a plurality of portions of the control object from the captured images, including:

calculating three co-planar tangents to observed edges of the control object from the captured images; and

fitting a circle to the control object using at least the three co-planar tangents.

16. The non-transitory computer readable medium of claim 15 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

for a complex control object model that includes a palm and multiple fingers, applying the determining and fitting a circle actions of claim 15 to construct multiple fingers of control object appendages; and

fitting cross sections of a palm to observed edges of the palm as captured in the images.

17. The non-transitory computer readable medium of claim 15 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

repeatedly applying the determining and fitting a circle actions of claim 15 over time; and

calculating motion of the control object over time based on differences between modeled locations of the control object.

18. The non-transitory computer readable medium of claim 16 , wherein tracking motion of the control object while making a gesture in a 3D space further includes:

repeatedly applying the determining and fitting a circle and fitting cross sections actions of claim 16 overtime; and

calculating motion of a complex control object over time based on differences between modeled locations of a complex control object.

19. A method for executing a gesture command, the method including:

receiving a result of analyzing a gesture;

interpreting as an instruction the result of analyzing a gesture; wherein the result of analyzing a gesture is obtained from another program executing on a processor from analysis of differences in positions of fit closed curves fit to observed edges of a control object to track motion of the control object while making a gesture in a 3D space including repeatedly:

obtaining captured images of a control object moving in 3D space captured from at least two geometrically distinct predetermined vantages;

calculating observed edges of the control object from the captured images;

fitting closed curves to the observed edges of the control object as captured in the captured images by selecting a closed curve from a family of similar closed curves that fit the observed edges of the control object as captured using an assumed parameter; and

whenever the control object is a complex control object that includes a palm and multiple fingers, applying actions including obtaining captured images, calculating observed edges, and fitting closed curves to construct multiple fingers of control object appendages; and

fitting cross sections of a palm to observed edges of a palm as captured in the captured images.

Assignments (7)
SECURITY INTEREST Recorded Apr 6, 2026
From: SIM IP HXR LLC
To: UNITY MASTER LLC SERIES XIX
Reel/Frame 075365/0907 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2026
From: ULTRAHAPTICS IP TWO LIMITED
To: SIM IP HXR LLC
Reel/Frame 075127/0545 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 16, 2026
From: ULTRAHAPTICS LIMITED; ULTRAHAPTICS IP LIMITED; ULTRAHAPTICS IP TWO LIMITED; ULTRALEAP LIMITED
To: SIM IP HXR LLC
Reel/Frame 074403/0573 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2022
From: HOLZ, DAVID
To: LEAP MOTION, INC.
Reel/Frame 061198/0457 →
CHANGE OF NAME Recorded Sep 23, 2022
From: OCUSPEC
To: LEAP MOTION, INC.
Reel/Frame 061535/0515 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2022
From: LEAP MOTION, INC.
To: LMI LIQUIDATING CO. LLC
Reel/Frame 061198/0498 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2022
From: LMI LIQUIDATING CO. LLC
To: ULTRAHAPTICS IP TWO LIMITED
Reel/Frame 061198/0530 →
Continuity (7)
Continuation 15953320 · Apr 13, 2018
Continuation 14723370 · May 27, 2015
Continuation 13724357 · Dec 21, 2012
Continuation In Part 13414485 · Mar 7, 2012
Provisional Application 61724091 · Nov 8, 2012
Provisional Application 61587554 · Jan 17, 2012
Related Publication 20200400428A1 · Dec 24, 2020
Cited By (2)
US 12,412,285 US 12,693,717