IP Library Granted Patent US 9,495,801
Granted Patent B2
US 9,495,801 · App. 14/267,711 · Granted Nov 15, 2016

Pose tracking an augmented reality device

Inventors: Michael John Ebstyne (Seattle, WA); Frederik Schaffalitzky (Bellevue, WA); Drew Steedly (Redmond, WA); Calvin Chan (Issaquah, WA); Ethan Eade (Seattle, WA); Alex Kipman (Redmond, WA); Georg Klein (Seattle, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
G06T19/006G02B27/017G06F3/011G06F3/012G06F3/038G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,495,801
App. No.
14/267,711
Filed
May 1, 2014
Granted
Nov 15, 2016
Kind
B2
Examiner
HE, YINGCHUN
Art Unit
2613
USPC
345/633
Abstract

An augmented reality device including a plurality of sensors configured to output pose information indicating a pose of the augmented reality device. The augmented reality device further includes a band-agnostic filter and a band-specific filter. The band-specific filter includes an error correction algorithm configured to receive pose information as filtered by the band-agnostic filter and reduce a tracking error of the pose information in a selected frequency band. The augmented reality device further includes a display engine configured to position a virtual object on a see-through display as a function of the pose information as filtered by the band-agnostic filter and the band-specific filter.

Claims (41)

1. An augmented reality device, comprising:

a plurality of sensors configured to output pose information indicating a pose of the augmented reality device;

a band-agnostic filter configured to receive pose information from the plurality of sensors and to reduce an all-band tracking error characterized by an entire frequency band of the pose information;

a band-specific filter configured to receive pose information as filtered by the band-agnostic filter, the band-specific filter configured to reduce a tracking error of the pose information in a selected frequency band that is less than the entire frequency band; and

a display engine configured to position a virtual object on a see-through display as a function of the pose information as filtered by the band-agnostic filter and the band-specific filter.

2. The augmented reality device of claim 1 , wherein the plurality of sensors include one or more of a gyroscope, a magnetometer, an accelerometer, and an image sensor.

3. The augmented reality device of claim 1 , wherein the band-agnostic filter uses a sensor fusion algorithm configured to:

receive the pose information from each of the plurality of sensors;

combine the pose information from each of the plurality of sensors; and

reduce the all-band tracking error of the combined pose information.

4. The sensor fusion algorithm of claim 3 , wherein the sensor fusion algorithm is configured to reduce a root-mean-square tracking error of the pose information.

5. The augmented reality device of claim 1 , wherein the band-agnostic filter uses an Extended Kalman filter.

6. The augmented reality device of claim 1 , wherein the band-specific filter uses a transfer function algorithm configured to:

receive pose information as filtered by the band-agnostic filter;

transfer the tracking error of the received pose information of the pose information from a time domain to a frequency domain; and

reduce the tracking error of the pose information within the selected frequency band.

7. The augmented reality device of claim 1 , wherein the band-specific filter uses an Extended Kalman filter.

8. The augmented reality device of claim 1 , wherein the band-specific filter uses a time constant to define the selected frequency band.

9. A method of augmented reality comprising:

receiving pose information from a plurality of sensors;

processing the pose information with a band-agnostic filter configured to reduce an all-band tracking error characterized by an entire frequency band of the pose information;

processing the pose information from the band-agnostic filter with a band-specific filter configured to reduce a tracking error of the pose information in a selected frequency band that is less than the entire frequency band; and

displaying a virtual object on a see-through display, such that the position of the virtual object on the see-through display is a function of the pose information as filtered by the band-agnostic filter and the band-specific filter.

10. The method of augmented reality of claim 9 , wherein the band-agnostic filter includes a sensor fusion algorithm configured to:

receive the pose information from each of the plurality of sensors;

combine the pose information from each of the plurality of sensors; and

reduce the all-band tracking error of the combined pose information.

11. The method of claim 10 , wherein the sensor fusion algorithm is configured to reduce a root-mean-square tracking error of the pose information.

12. The method of augmented reality of claim 9 , wherein the band-agnostic filter includes an Extended Kalman filter.

13. The method of augmented reality of claim 9 , wherein the band-specific filter includes a transfer function algorithm configured to:

receive pose information as filtered by the band-agnostic filter;

transfer the tracking error of the pose information from a time domain to a frequency domain; and

reduce the tracking error of the pose information within the selected frequency band.

14. The method of augmented reality of claim 9 , wherein the band-specific filter includes an Extended Kalman filter.

15. The method of augmented reality of claim 9 , wherein the band-specific filter includes an error correction algorithm uses a time constant to define the selected frequency band.

16. A head-mounted display device comprising:

a wearable see-through display;

a plurality of sensors configured to output pose information indicating a tracked pose of the head-mounted display device; and

a display engine configured to position a virtual object on a see-through display as a function of the tracked pose of the head-mounted display device such that a perceived positional difference between the virtual object and a real-world object viewed through the see-though display is filtered by a two stage filter including a band-agnostic filter configured to reduce an all-band tracking error characterized by an entire frequency band of the pose information and a band-specific filter configured to receive filtered pose information from the band-agnostic filter and reduce a tracking error of the filtered pose information in a specified frequency band that is less than the entire frequency band.

17. The head-mounted display device of claim 16 , wherein the display engine uses a first, band-agnostic Extended Kalman filter and a second, band-specific Extended Kalman filter.

18. The head-mounted display device of claim 16 , wherein the plurality of sensors include one or more of a gyroscope, a magnetometer, an accelerometer, and an image sensor.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2016
From: EBSTYNE, MICHAEL JOHN; SCHAFFALITZKY, FREDERIK; STEEDLY, DREW; CHAN, CALVIN; EADE, ETHAN; KIPMAN, ALEX; KLEIN, GEORG
To: MICROSOFT CORPORATION
Reel/Frame 039877/0094 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
Continuity (1)
Related Publication 20150317833A1 · Nov 5, 2015