IP Library Granted Patent US 10,824,244
Granted Patent B2
US 10,824,244 · App. 16/277,761 · Granted Nov 3, 2020

Systems and methods for transitioning between modes of tracking real-world objects for artificial reality interfaces

Inventors: Kenrick Cheng-Kuo Kin (Redmond, WA); Maxime Ouellet (Kirkland, WA)
Assignee: Facebook Technologies, LLC
G06F3/0304A63F13/211A63F13/212G06F3/011G06F3/017A63F2300/8082
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,824,244
App. No.
16/277,761
Granted
Nov 3, 2020
Kind
B2
Abstract

The disclosed computer-implemented method may include tracking (1) a position of a primary real-world object within a real-world environment via a primary tracking method, and (2) a position of a secondary real-world object within the real-world environment via a secondary tracking method. The method may further include presenting (1) a primary virtual object at a position within an artificial environment corresponding to the tracked position of the primary real-world object, and (2) a secondary virtual object at a position within the artificial environment corresponding to the tracked position of the secondary real-world object. The method may further include (1) detecting an interaction of the primary real-world object with the secondary real-world object, and (2) transitioning to tracking the position of the primary real-world object via the secondary tracking method. Various other methods, systems, and computer-readable media are also disclosed.

Claims (98)

1. A computer-implemented method comprising:

tracking:

a position of a primary real-world object within a real-world environment via a primary tracking method; and

a position of a secondary real-world object within the real-world environment via a secondary tracking method;

presenting:

a primary virtual object that represents the primary real-world object at a position within an artificial environment corresponding to the position of the primary real-world object within the real-world environment; and

a secondary virtual object that represents the secondary real-world object at a position within the artificial environment corresponding to the position of the secondary real-world object within the real-world environment;

detecting:

an interaction of the primary real-world object with the secondary real-world object; and

an additional interaction of the primary real-world object with the secondary real-world object;

determining a proximity of the primary real-world object to the secondary real-world object by:

determining a position of the primary real-world object based on the primary tracking method;

determining a position of the secondary real-world object based on the secondary tracking method; and

comparing the position of the primary real-world object with the position of the secondary real-world object; and

transitioning:

from tracking the position of the primary real-world object within the real-world environment via the primary tracking method to tracking the position of the primary real-world object within the real-world environment via the secondary tracking method in response to detecting the interaction of the primary real-world object with the secondary real-world object; and

from tracking the position of the primary real-world object within the real-world environment via the secondary tracking method to tracking the position of the primary real-world object within the real-world environment via the primary tracking method in response to detecting the additional interaction of the primary real-world object with the secondary real-world object.

2. The computer-implemented method of claim 1 , wherein the primary tracking method comprises a computer vision tracking method and the secondary tracking method comprises at least one of:

an optical tracking method;

a simultaneous localization and mapping (SLAM) tracking method; or

an inertial tracking method.

3. The computer-implemented method of claim 1 , further comprising adjusting, in response to transitioning from tracking the position of the primary real-world object within the real-world environment via the primary tracking method to tracking the position of the primary real-world object within the real-world environment via the secondary tracking method, an appearance of at least one of:

the primary virtual object; or

the secondary virtual object.

4. The computer-implemented method of claim 1 , wherein detecting the interaction of the primary real-world object with the secondary real-world object comprises determining that the proximity of the primary real-world object to the secondary real-world object is less than a predetermined threshold.

5. The computer-implemented method of claim 1 , further comprising adjusting, based on the proximity of the primary real-world object to the secondary real-world object, an appearance of at least one of:

the primary virtual object; or

the secondary virtual object.

6. The computer-implemented method of claim 1 , further comprising presenting, in response to transitioning from tracking the position of the primary real-world object within the real-world environment via the primary tracking method to tracking the position of the primary real-world object within the real-world environment via the secondary tracking method, at the position within the artificial environment corresponding to the position of the primary real-world object within the real-world environment, a unified virtual object that represents both the primary real-world object and the secondary real-world object.

7. The computer-implemented method of claim 1 , wherein:

the primary real-world object comprises a hand of a user; and

the secondary real-world object comprises an artificial reality controller device.

8. The computer-implemented method of claim 1 , wherein:

the secondary real-world object comprises a touch sensor; and

detecting the interaction of the primary real-world object with the secondary real-world object comprises detecting, via the touch sensor, a touch of the secondary real-world object by the primary real-world object.

9. The computer-implemented method of claim 1 , further comprising determining that the primary real-world object comprises one of:

a left hand of a user; or

a right hand of the user.

10. The computer-implemented method of claim 9 , wherein:

the secondary real-world object comprises an artificial reality controller device configured to be operated by the user via one of:

the left hand of the user; or

the right hand of the user; and

the computer-implemented method further comprises presenting a notification to the user upon detecting the interaction of the primary real-world object with the secondary real-world object and when at least one of:

the artificial reality controller device is configured to be operated by the right hand of the user and upon determining that the primary real-world object comprises the left hand of the user; or

the artificial reality controller device is configured to be operated by the left hand of the user and upon determining that the primary real-world object comprises the right hand of the user.

11. A system comprising:

a tracking module, stored in memory, that tracks:

a position of a primary real-world object within a real-world environment via a primary tracking method; and

a position of a secondary real-world object within the real-world environment via a secondary tracking method;

a presenting module, stored in memory, that presents:

a primary virtual object that represents the primary real-world object at a position within an artificial environment corresponding to the position of the primary real-world object within the real-world environment; and

a secondary virtual object that represents the secondary real-world object at a position within the artificial environment corresponding to the position of the secondary real-world object within the real-world environment;

a detecting module, stored in memory, that detects:

an interaction of the primary real-world object with the secondary real-world obiect;

an additional interaction of the primary real-world object with the secondary real-world object; and

a proximity of the primary real-world object to the secondary real-world object By:

determining a position of the primary real-world object based on the primary tracking method;

determining a position of the secondary real-world object based on the secondary tracking method; and

comparing the position of the primary real-world object with the position of the secondary real-world object;

a transitioning module, stored in memory, that transitions:

from tracking the position of the primary real-world object within the real-world environment via the primary tracking method to tracking the position of the primary real-world object within the real-world environment via the secondary tracking method in response to detecting the interaction of the primary real-world object with the secondary real-world object; and

from tracking the position of the primary real-world object within the real-world environment via the secondary tracking method to tracking the position of the primary real-world object within the real-world environment via the primary tracking method in response to detecting the additional interaction of the primary real-world object with the secondary real-world object; and

at least one physical processor that executes the tracking module, the presenting module, the detecting module, and the transitioning module.

12. The system of claim 11 , wherein the presenting module further adjusts, in response to the transitioning module transitioning from tracking the position of the primary real-world object within the real-world environment via the primary tracking method to tracking the position of the primary real-world object within the real-world environment via the secondary tracking method, an appearance of at least one of:

the primary virtual object; or

the secondary virtual object.

13. The system of claim 11 , wherein the detecting module detects the interaction of the primary real-world object with the secondary real-world object by determining that the proximity of the primary real-world object to the secondary real-world object is less than a predetermined threshold.

14. The system of claim 11 , wherein the presenting module adjusts, based on the proximity of the primary real-world object to the secondary real-world object, an appearance of at least one of:

the primary virtual object; or

the secondary virtual object.

15. The system of claim 11 , wherein:

the secondary real-world object comprises an artificial reality controller device configured to be operated by a user via one of:

a left hand of the user; or

a right hand of the user;

the detecting module further determines that the primary real-world object comprises one of:

the left hand of the user; or

the right hand of the user; and

the presenting module further presents a notification to the user upon the detecting module detecting the interaction of the primary real-world object with the secondary real-world object and when at least one of:

the artificial reality controller device is configured to be operated by the right hand of the user and upon the detecting module determining that the primary real-world object comprises the left hand of the user; or

the artificial reality controller device is configured to be operated by the left hand of the user and upon the detecting module determining that the primary real-world object comprises the right hand of the user.

16. A non-transitory computer-readable medium comprising instructions that, when executed by at least one processor of a computing system, cause the computing system to:

track:

a position of a primary real-world object within a real-world environment via a primary tracking method; and

a position of a secondary real-world object within the real-world environment via a secondary tracking method;

present:

a primary virtual object that represents the primary real-world object at a position within an artificial environment corresponding to the position of the primary real-world object within the real-world environment; and

a secondary virtual object that represents the secondary real-world object at a position within the artificial environment corresponding to the position of the secondary real-world object within the real-world environment;

detect:

an interaction of the primary real-world object with the secondary real-world object; and

an additional interaction of the primary real-world object with the secondary real-world object;

determine a proximity of the primary real-world object to the secondary real-world object by:

determining a position of the primary real-world object based on the primary tracking method;

determining a position of the secondary real-world object based on the secondary tracking method; and

comparing the position of the primary real-world object with the position of the secondary real-world object; and

transition:

from tracking the position of the primary real-world object within the real-world environment via the primary tracking method to tracking the position of the primary real-world object within the real-world environment via the secondary tracking method in response to detecting the interaction of the primary real-world object with the secondary real-world object; and

from tracking the position of the primary real-world object within the real-world environment via the secondary tracking method to tracking the position of the primary real-world object within the real-world environment via the primary tracking method in response to detecting the additional interaction of the primary real-world object with the secondary real-world object.

17. The computer-implemented method of claim 1 , wherein the secondary virtual object is at least partially visible through the primary virtual object within the artificial environment.

Assignments (2)
CHANGE OF NAME Recorded May 26, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060199/0876 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2019
From: KIN, KENRICK CHENG-KUO; OUELLET, MAXIME
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 048607/0193 →
Continuity (2)
Provisional Application 62769542 · Nov 19, 2018
Related Publication 20200159337A1 · May 21, 2020
Cited By (1)
US 12,482,193