IP Library Granted Patent US 10,664,993
Granted Patent B1
US 10,664,993 · App. 15/918,695 · Granted May 26, 2020

System for determining a pose of an object

Inventors: Vikas M. Reddy (Boulder, CO); Jeffrey Roger Powers (San Francisco, CA); Paul Jakob Schroeder (Boulder, CO); Chris Slaughter (Boulder, CO); Ganesh Mallya (Boulder, CO)
Assignee: Occipital, Inc.
G06T7/75G06F3/0425G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,664,993
App. No.
15/918,695
Granted
May 26, 2020
Kind
B1
Abstract

A system configured to determine a pose of a physical object in a physical environment and to utilize the pose as an input to control or manipulate a virtual environment or mixed reality environment. In some cases, the system may be capture image data of a physical object having a constellation or pattern on the external source. The system may analyze the image data to identify points associated with the constellation or pattern and to determine the pose based on the identified points and a stored object model associated with the physical object.

Claims (58)

1. A system comprising:

a display for presenting a virtual environment to a user;

one or more image components for capturing image data associated with a physical environment surrounding the user;

one or more processors;

non-transitory computer-readable media storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to:

identify image points within the image data, the image points corresponding to constellation points on an exterior of a physical object;

determining a set of candidates based at least in part on the image points;

determining a pose of the physical object based at least in part on the set of candidates and an object model, the object model being a three-dimensional representation of the physical object and including model points corresponding to the constellation points on the exterior of the physical object; and

outputting the pose to an application associated with the virtual environment presented on the display.

2. The system as recited in claim 1 , wherein the non-transitory computer-readable media stores additional computer-executable instructions, which when executed by the one or more processors cause the one or more processors to assigning a class to at least one of the image points and wherein the set of candidates is determined at least in part based on the class of the at least one of the image points.

3. The system as recited in claim 1 , wherein determining the set of candidates is based at least in part on a previous pose of the physical object determined with respect to previous image data.

4. The system as recited in claim 1 , further comprising:

one or more communication interfaces to receive orientation data from the physical object; and

wherein the set of candidates is determined based at least in part on the orientation data.

5. The system as recited in claim 1 , further comprising:

one or more measurement units configured to capture orientation data associated with the system; and

wherein the set of candidates is determined based at least in part on the orientation data.

6. The system as recited in claim 5 , wherein the image points identified include a set of two or three image points.

7. The system as recited in claim 5 , wherein the one or more measurement units include at least one of the following:

an internal measurement unit (IMU);

an accelerometers;

a gyroscope; or

a magnetometer.

8. The system as recited in claim 1 , wherein the image component is configured to capture the image data from substantially a same perspective as a perspective of the virtual environment.

9. The system as recited in claim 1 , wherein the image points identified include a set of four image points.

10. A method comprising:

receiving two-dimensional image data, the two-dimensional image data including data associated with a physical object;

identify image points within the data associated with the physical object, the image points corresponding to constellation points on an exterior of the physical object;

determining a set of candidates based at least in part on the image points, each candidate including at least one image point to model point relationship;

determining a pose of the physical object based at least in part on the set of candidates; and

outputting the pose to an application associated with a virtual environment displayed to a user.

11. The method as recited in claim 10 , wherein determine the pose of the physical object includes:

determining a number of inliers associated with individual ones of the set of candidates;

selecting a first candidate from the set of candidates based at least in part on the number of inliers associated with the first candidate; and

determining the pose of the object based at least in part on data associated with the first candidate.

12. The method as recited in claim 10 , further comprising:

receiving orientation data from the physical object; and

wherein the image points identified includes a set of two or three image points and the set of candidates is determined based at least in part on the two or three image points and the orientation data.

13. The method as recited in claim 10 , wherein determining the set of candidates is based at least in part on a previous pose of the physical object determined with respect to previous image data.

14. The method as recited in claim 10 , wherein:

the virtual environment is displayed on a display of an image system; and

the set of candidates is determined based at least in part on orientation data associated with the image system.

15. A system comprising:

a handheld device including:

a plurality of constellation points arranged on an exterior surface of the device, the exterior surface of the device exposed to a physical environment;

an inertial measurement unit coupled to the handled device and configured to capture orientation data associated with the handheld device; and

one or more communication interfaces to send the orientation data to an image device; and

wherein the image device includes non-transitory computer-readable media storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform operations including:

receiving two-dimensional image data, the two-dimensional image data including data associated with the handheld device;

identify image points within the two-dimensional image data corresponding to the plurality of constellation points;

determining a set of candidates based at least in part on the image points;

determining a pose of the physical object based at least in part on the set of candidates; and

causing an output device to display the pose in conjunction with an application associated with a virtual environment to a user.

16. The system as recited in claim 15 , wherein individual constellation points of the plurality of constellation points are light sources and the light sources are configured to activate at high power for predetermined period of time, the predetermined period of time substantially synchronized with an exposure interval of a remote image component.

17. The system as recited in claim 15 , wherein the plurality of constellation points are light emitting devices (LEDs).

18. The system as recited in claim 15 , wherein the plurality of constellation points are formed by applying a coating to an exterior of the device, the coating undetectable in a visible spectrum but detectable in an infrared spectrum.

19. The system as recited in claim 15 , further comprising a second measurement unit configured to capture additional orientation data associated with the device, the second measurement unit different than the inertial measurement unit.

20. The system as recited in claim 15 , further comprising a physical user input device and wherein a signal representative of a user selection of the physical user input device is sent to the image system via the communication interface.

Assignments (4)
SECURITY INTEREST Recorded Apr 21, 2026
From: XRPRO, LLC
To: DANLAW, INC.
Reel/Frame 075436/0662 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 22, 2022
From: OCCIPITAL, INC.
To: XRPRO, LLC
Reel/Frame 061296/0846 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 22, 2022
From: OCCIPITAL, INC.
To: XRPRO, LLC
Reel/Frame 061408/0869 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2019
From: REDDY, VIKAS M.; POWERS, JEFFREY ROGER; SCHROEDER, PAUL JAKOB; SLAUGHTER, CHRIS; MALLYA, GANESH
To: OCCIPITAL, INC.
Reel/Frame 048133/0752 →
Continuity (2)
Provisional Application 62470767 · Mar 13, 2017
Provisional Application 62620586 · Jan 23, 2018
Cited By (2)
US 12,437,438 US 12,487,663