IP Library Granted Patent US 11,662,805
Granted Patent B2
US 11,662,805 · App. 17/301,655 · Granted May 30, 2023

Periodic parameter estimation for visual-inertial tracking systems

Inventors: Georg Halmetschlager-Funek (Vienna, AT); Matthias Kalkgruber (Vienna, AT); Daniel Wolf (Mödling, AT); Jakob Zillner (Absdorf, AT)
Assignee: Snap Inc.
G06F3/012G06F3/038H04L67/131G06F2203/0383
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,662,805
App. No.
17/301,655
Filed
Apr 9, 2021
Granted
May 30, 2023
Kind
B2
Art Unit
2624
USPC
345/156
Abstract

A method for calibrating a visual-inertial tracking system is described. A device operates the visual-inertial tracking system without receiving a tracking request from a virtual object display application. In response to operating the visual-inertial tracking system, the device accesses sensor data from sensors at the device. The device identifies, based on the sensor data, a first calibration parameter value of the visual-inertial tracking system and stores the first calibration parameter value. The system detects a tracking request from the virtual object display application. In response to the tracking request, the system accesses the first calibration parameter value and determines a second calibration parameter value from the first calibration parameter value.

Claims (64)

1. A method for calibrating a visual-inertial tracking system comprising:

detecting, at a head-mounted device, that a virtual object display application that is configured to operate at the head-mounted device is not requesting tracking operations from the visual-inertial tracking system of the head-mounted device;

in response to the detection, operating, at the head-mounted device, the visual-inertial tracking system

by accessing sensor data from a plurality of sensors only at the head-mounted device;

identifying, at the head-mounted device, a first calibration parameter value of the visual-inertial tracking system based on the sensor data from the plurality of sensors only at the head-mounted device;

storing, in a storage of the head-mounted device, the first calibration parameter value;

detecting an operation of the virtual object display application at the head-mounted device by detecting a tracking request from the virtual object display application to the visual-inertial tracking system; and

in response to detecting the tracking request, accessing, at the storage of the head-mounted device, the first calibration parameter value and determining, at the head-mounted device, a second calibration parameter value of the virtual-inertial tracking system from the first calibration parameter value.

2. The method of claim 1 , wherein operating the visual-inertial tracking system further comprises:

periodically accessing the sensor data from the plurality of sensors,

wherein the visual-inertial tracking system operates independently from the virtual object display application.

3. The method of claim 1 , further comprising:

detecting a calibration trigger event at the head-mounted device,

wherein operating the visual-inertial tracking system is in response to detecting the calibration trigger event.

4. The method of claim 3 , wherein the trigger event comprises at least one of a temperature change exceeding a temperature threshold, an accelerometer sensor value exceeding an accelerometer threshold value, a battery level of the device exceeding a battery threshold value, a user-activation of the head-mounted device, or a detection that the head-mounted device is worn by a user.

5. The method of claim 1 , wherein identifying the first calibration parameter value is based on a convergence between a first virtual object data point location and a second virtual object data point location, the first virtual object data point location being determined based on the sensor data that are adjusted with the first parameter value, the second virtual object data point location being determined based on the sensor data that are adjusted with a default calibration parameter value of the head-mounted device.

6. The method of claim 1 , further comprising:

turning off the visual-inertial tracking system after storing the first calibration parameter value; and

activating the visual-inertial tracking system in response to detecting an operation of the virtual object display application.

7. The method of claim 1 , further comprising:

calibrating the visual-inertial tracking system with the first calibration parameter value before operating the virtual object display application.

8. The method of claim 1 , further comprising:

calibrating the visual-inertial tracking system with the second calibration parameter value after detecting the tracking request from the virtual object display application.

9. The method of claim 1 , wherein determining the second calibration parameter value further comprises:

replacing a starting value comprising a default calibration value with the first calibration parameter value; and

performing a calibration of the visual-inertial tracking system starting with the starting value.

10. The method of claim 1 , wherein storing the first calibration parameter value further comprises:

storing the first calibration value in at a server.

11. A computing apparatus comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the apparatus to perform operations comprising:

detect, at a head-mounted device, that a virtual object display application that is configured to operate at the head-mounted device is not requesting tracking operations from the visual-inertial tracking system of the head-mounted device;

in response to the detection, operate, at the head-mounted device, the visual-inertial tracking system

by accessing sensor data from a plurality of sensors only at the head-mounted device;

identify, at the head-mounted device, a first calibration parameter value of the visual-inertial tracking system based on the sensor data from the plurality of sensors only at the head-mounted device;

store, in a storage of the head-mounted device, the first calibration parameter value;

detect an operation of the virtual object display application at the head-mounted device by detecting a tracking request from the virtual object display application to the visual-inertial tracking system; and

in response to detecting the tracking request, access, at the storage of the head-mounted device, the first calibration parameter value and determining, at the head-mounted device, a second calibration parameter value of the virtual-inertial tracking system from the first calibration parameter value.

12. The computing apparatus of claim 11 , wherein operating the visual-inertial track system further comprises:

periodically access the sensor data from the plurality of sensors,

wherein the visual-inertial track system operates independently from the virtual object display application.

13. The computing apparatus of claim 11 , wherein the instructions further configure the apparatus to:

detect a calibration trigger event at the head-mounted device,

wherein operating the visual-inertial track system is in response to detecting the calibration trigger event.

14. The computing apparatus of claim 13 , wherein the trigger event comprises at least one of a temperature change exceeding a temperature threshold, an accelerometer sensor value exceeding an accelerometer threshold value, a battery level of the head-mounted device exceeding a battery threshold value, a user-activation of the head-mounted device, or a detection that the head-mounted device is worn by a user.

15. The computing apparatus of claim 11 , wherein identifying the first calibration parameter value is based on a convergence between a first virtual object data point location and a second virtual object data point location, the first virtual object data point location being determined based on the sensor data that are adjusted with the first parameter value, the second virtual object data point location being determined based on the sensor data that are adjusted with a default calibration parameter value of the head-mounted device.

16. The computing apparatus of claim 11 , wherein the instructions further configure the apparatus to:

turn off the visual-inertial tracking system after storing the first calibration parameter value; and

activating the visual-inertial tracking system in response to detecting an operation of the virtual object display application.

17. The computing apparatus of claim 11 , wherein the instructions further configure the apparatus to:

calibrate the visual-inertial tracking system with the first calibration parameter value before operating the virtual object display application.

18. The computing apparatus of claim 11 , wherein the instructions further configure the apparatus to:

calibrate the visual-inertial tracking system with the second calibration parameter value after detecting the tracking request from the virtual object display application.

19. The computing apparatus of claim 11 , wherein determining the second calibration parameter value further comprises:

replace a starting value comprising a default calibration value with the first calibration parameter value; and

perform a calibration of the visual-inertial tracking system starting with the starting value.

20. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:

detect, at a head-mounted device, that a virtual object display application that is configured to operate at the head-mounted device is not requesting tracking operations from the visual-inertial tracking system of the head-mounted device;

in response to the detection, operate, at the head-mounted device, the visual-inertial tracking system

by accessing sensor data from a plurality of sensors only at the head-mounted device;

identify, at the head-mounted device, a first calibration parameter value of the visual-inertial tracking system based on the sensor data from the plurality of sensors only at the head-mounted device;

store, in a storage of the head-mounted device, the first calibration parameter value;

detect an operation of the virtual object display application at the head-mounted device by detecting a tracking request from the virtual object display application to the visual-inertial tracking system; and

in response to detecting the tracking request, access, at the storage of the head-mounted device, the first calibration parameter value and determining, at the head-mounted device, a second calibration parameter value of the virtual-inertial tracking system from the first calibration parameter value.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 14, 2023
From: HALMETSCHLAGER-FUNEK, GEORG; KALKGRUBER, MATTHIAS; WOLF, DANIEL; ZILLNER, JAKOB
To: SNAP INC.
Reel/Frame 063323/0903 →
Continuity (2)
Provisional Application 63131981 · Dec 30, 2020
Related Publication 20220206565A1 · Jun 30, 2022
Cited By (2)
US 12,210,672 US 12,579,690