IP Library Granted Patent US 9,767,609
Granted Patent B2
US 9,767,609 · App. 14/179,384 · Granted Sep 19, 2017

Motion modeling in visual tracking

Inventors: Ethan Eade (Seattle, WA); Michael John Ebstyne (Seattle, WA); Frederick Schaffalitzky (Bellevue, WA); Drew Steedly (Redmond, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
G06T19/006G02B27/017G06F3/012G06F3/013G06F3/017G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,767,609
App. No.
14/179,384
Granted
Sep 19, 2017
Kind
B2
Abstract

Embodiments are disclosed that relate to determining a pose of a device. One disclosed embodiment provides a method comprising receiving sensor information from one or more sensors of the device, and selecting a motion-family model from a plurality of different motion-family models based on the sensor information. The method further comprises providing the sensor information to the selected motion-family model and outputting an estimated pose of the device according to the selected motion-family model.

Claims (41)

1. A method of assessing a pose of a device, the method comprising:

receiving, from one or more sensors of the device, sensor information including a change in location of the device along three orthogonal axes and a change in orientation of the device along the three orthogonal axes;

using a pre-configured motion behavior classifier to determine a physical state of a user of the device based on the change in location of the device and the change in orientation of the device;

selecting a motion-family model from a plurality of different pre-configured motion-family models based on the physical state of the user;

providing the sensor information to the selected motion-family model;

receiving an estimated pose of the device from the selected motion-family model, wherein the selected motion-family model interprets the sensor information in accordance with a family of motion associated with the device for the selected motion-family model, the family of motion including an acceleration profile for the device, and the selected motion-family model matching one or more measurements of the device to a selected trajectory of the device; and

displaying content on the device based on the estimated pose of the device.

2. The method of claim 1 , wherein the device is a head-mounted display device configured to output the displayed content to a wearer of the device, the displayed content being mapped, based on the estimated pose, to changing locations of a real-world environment viewable through the head-mounted display device as the location and orientation of the head-mounted display device changes.

3. The method of claim 2 , further comprising adjusting one or more display properties for the displayed content based on the estimated pose of the device.

4. The method of claim 1 , wherein outputting the estimated pose of the device further comprises providing the sensor information and an output of the selected motion-family model to a pose filter and outputting the estimated pose of the device from the pose filter.

5. The method of claim 1 , wherein the sensor information indicates a physical state of a wearer of the device, the physical state indicating one or more of a position, orientation, and movement characteristic of the wearer.

6. The method of claim 1 , wherein selecting the motion-family model comprises classifying a family of motion exhibited by the device and comparing the classified family of motion to one or more families of motion associated with the plurality of different pre-configured motion-family models.

7. The method of claim 6 , wherein classifying the family of motion exhibited by the device comprises determining the family of motion based on a previously-output estimated pose of the device.

8. A head-mounted display device comprising:

a sensor;

a logic machine; and

a storage machine, the storage machine holding instructions executable by the logic machine to:

receive, from the sensor, sensor information including a change in location of the head-mounted display device along three orthogonal axes and a change in orientation of the head-mounted display device along the three orthogonal axes;

use a pre-configured motion behavior classifier to determine a physical state of a user of the head-mounted display device based on the change in location of the head-mounted display device and the change in orientation of the head-mounted display device;

select a motion-family model from a plurality of pre-configured motion-family models based on the physical state of a wearer of the head-mounted display device, each of the plurality of pre-configured motion-family models associated with a respective family of motion for the device, each respective family of motion including a different acceleration profile, and each pre-configured motion-family model matching one or more measurements of the sensor to a different trajectory of head movement indicating predefined expected movement of the head-mounted display device for the associated physical state of the user;

provide the sensor information and an output of the selected motion-family model to a pose filter;

output, from the pose filter, an estimated position and orientation of the head-mounted display device based on the sensor information and the selected motion-family model; and

display content on the head-mounted display device based on the estimated pose of the head-mounted display device.

9. The head-mounted display device of claim 8 , wherein the physical state of the wearer is determined based on the sensor information from the sensor.

10. The head-mounted display device of claim 8 , wherein the sensor information includes one or more measurements of a position and/or orientation of a head of the wearer.

11. The head-mounted display device of claim 10 , wherein the motion-family model matches the one or more measurements to a selected trajectory of head movement based on the physical state of the wearer.

12. The head-mounted display device of claim 8 , wherein the pose filter includes a Kalman filter.

13. The head-mounted display device of claim 8 , wherein selecting the motion-family model from the plurality of pre-configured motion-family models further comprises sending the sensor information and a previously-output estimated position and orientation to a pre-configured motion behavior classifier of the head-mounted display device.

14. The head-mounted display device of claim 13 , wherein the sensor information is provided to the pose filter and the pre-configured motion behavior classifier.

15. The head-mounted display device of claim 13 , wherein the pre-configured motion behavior classifier is used to analyze the sensor information and the previously-output estimated position and orientation to select the motion-family model.

16. The head-mounted display device of claim 15 , further comprising changing the motion-family model if the selected motion-family model is different than a current motion-family model.

17. A hardware memory device holding instructions executable by a logic machine to:

collect sensor information from one or more sensors of a head-mounted display;

send the sensor information to a pre-configured motion behavior classifier for determining a selected motion-family model from a plurality of different pre-configured motion-family models based on the sensor information;

in response to determining that the selected motion-family model is different than a previously-used motion-family model, replace the previously-used motion-family model with the selected motion-family model;

wherein each of the previously-used motion-family model and the selected motion-family model is used to adjust the sensor information to match one or more measurements indicating head movement to a selected trajectory of head movement for that motion-family model;

receive a time-varying pose estimate that is output based on an output of both the previously-used motion-family model and the selected motion-family model; and

display content on the head-mounted display based on the time-varying pose estimate of the head-mounted display.

18. The hardware memory device of claim 17 , wherein a first, earlier time-varying pose estimate is based on the previously-used motion-family model and a second, later time-varying pose estimate is based on the selected motion-family model.

19. The hardware memory device of claim 17 , wherein a first, earlier time-varying pose estimate is based on the previously-used motion-family model and a second, later time-varying pose estimate is based on an average of an output of the previously-used motion-family model and the selected motion-family model.

20. The hardware memory device of claim 17 , wherein one or more of the sensor information and the time-varying pose estimate is provided to a Kalman filter.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2016
From: EADE, ETHAN; EBSTYNE, MICHAEL JOHN; SCHAFFALITZKY, FREDERIK; STEEDLY, DREW
To: MICROSOFT CORPORATION
Reel/Frame 038836/0797 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
Continuity (1)
Related Publication 20150228118A1 · Aug 13, 2015