IP Library Granted Patent US 9,727,776
Granted Patent B2
US 9,727,776 · App. 14/288,287 · Granted Aug 8, 2017

Object orientation estimation

Inventors: Bhaven P. Dedhia (Redmond, WA); Yaron Eshet (Seattle, WA); Geoffrey J. Hulten (Lynnwood, WA)
Assignee: Microsoft Technology Licensing, LLC
G06K9/00268G06T7/75G06T2207/10024G06T2207/10028G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,727,776
App. No.
14/288,287
Granted
Aug 8, 2017
Kind
B2
Abstract

The description relates to estimating object orientation. One example includes determining a first estimate of object orientation using a first technique and image data. In this example, a second estimate of the object orientation can be determined using a second technique and the image data. The first estimate can be corrected with the second estimate to generate a corrected object orientation estimate which can be output.

Claims (41)

1. A method performed by a computing device, the method comprising:

differentiating an object from other portions of an image, the image comprising two-dimensional image data and depth data;

determining landmarks of the differentiated object;

determining a first estimate of object orientation of the differentiated object using a first technique that uses the depth data in determining a geometric model of the differentiated object;

determining a second estimate of the object orientation of the differentiated object using a second technique with the landmarks of the differentiated object;

correcting the first estimate using the second estimate to generate a corrected object orientation estimate; and

outputting the corrected object orientation estimate.

2. The method of claim 1 , wherein the geometric model includes a plane of the differentiated object.

3. The method of claim 1 , wherein the first technique comprises determining the landmarks and an up vector of the differentiated object.

4. The method of claim 1 , wherein the first estimate comprises estimates of angles of rotation relative to multiple axes of rotation of the differentiated object.

5. The method of claim 4 , wherein the multiple axes of rotation of the differentiated object correspond to pitch, roll, and yaw.

6. The method of claim 4 , wherein the second estimate of the object orientation includes another estimate of at least one of the angles of rotation.

7. The method of claim 6 , wherein the multiple axes of rotation of the differentiated object correspond to pitch, roll, and yaw and the second estimate corresponds to yaw.

8. The method of claim 1 , wherein the landmarks are determined from the two-dimensional image data.

9. The method of claim 1 , wherein the depth data includes time-of-flight depth data.

10. The method of claim 1 , further comprising assigning a weight to the second estimate according to a distance of the differentiated object from a camera that collects the depth data, wherein the weight is used to generate the corrected object orientation estimate.

11. The method of claim 1 , wherein the correcting includes determining an extent to which the first estimate is consistent with the second estimate.

12. The method of claim 1 , wherein the differentiated object is a body part of a user, a handheld object, a vehicle, or a computing device.

13. A system, comprising:

a processing device; and

a storage device storing computer-executable instructions which, when executed by the processing device, cause the processing device to:

obtain an image comprising depth data;

differentiate an object from other portions of the image;

determine landmarks of the differentiated object;

determine a first estimate of object orientation of the differentiated object using a first technique that uses the depth data to determine a geometric model of the differentiated object;

determine a second estimate of the object orientation of the differentiated object using a second technique that considers the landmarks of the differentiated object; and

correct the first estimate using the second estimate to generate a corrected object orientation estimate.

14. The system of claim 13 , wherein the image further comprises two-dimensional image data and the landmarks are determined from the two-dimensional image data.

15. The system of claim 13 , wherein an overall accuracy of the corrected object orientation estimate is greater than a first accuracy of the first estimate or a second accuracy of the second estimate.

16. The system of claim 13 , wherein the second technique is an iterative technique.

17. The system of claim 16 , wherein the computer-executable instructions further cause the processing device to adjust a number of iterations of the iterative technique according to available computational resources of the system.

18. A computer-executable memory device or storage device storing computer-executable instructions that, when executed by one or more processing devices, cause the one or more processing devices to perform acts comprising:

detecting an object in an image;

determining landmarks of the detected object;

obtaining a first estimate of object orientation of the detected object, the first estimate being provided by a first technique that determines a geometric model of the detected object;

obtaining a second estimate of the object orientation of the detected object, the second estimate being provided by a second technique that considers the landmarks of the detected object;

correcting the first estimate using the second estimate, based at least in part on an extent to which the first estimate is consistent with the second estimate, to generate a corrected object orientation estimate; and

outputting the corrected object orientation estimate.

19. The computer-executable memory device or storage device of claim 18 , wherein the detected object is a head of a user.

20. The computer-executable memory device or storage device of claim 19 , wherein the geometric model is a facial plane of the head.

21. The computer-executable memory device or storage device of claim 20 , wherein the first estimate of the object orientation is a rotation angle estimate determined using the facial plane of the head.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 27, 2014
From: DEDHIA, BHAVEN P.; ESHET, YARON; HULTEN, GEOFFREY J.
To: MICROSOFT CORPORATION
Reel/Frame 032970/0053 →
Continuity (1)
Related Publication 20150348269A1 · Dec 3, 2015