IP Library Granted Patent US 9,129,375
Granted Patent B1
US 9,129,375 · App. 13/455,949 · Granted Sep 8, 2015

Pose detection

Inventors: Qiang Liu (Cupertino, CA); Samuel Henry Chang (San Jose, CA)
Assignee: Rawles LLC
G06T7/0038G06T3/0068G06T7/0022G06T7/0046H04N7/54H04N9/31G06T19/00H04N9/28
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,129,375
App. No.
13/455,949
Granted
Sep 8, 2015
Kind
B1
Abstract

Techniques are described for determining the pose of an object based on a 3D point set representing the object. The 3D point set is rotated into a 2D coordinate system and a model object contour is aligned with the contour of the rotated point set using an iterative process. The aligned model object contour is then rotated back into the original 3D coordinate system, where its pose is assumed to represent the pose of the object.

Claims (64)

1. A system comprising:

one or more processors;

an imaging sensor;

a projector;

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:

observing, based at least in part on the image sensor, at least a portion of a display medium;

determining a point set corresponding to the observed display medium with respect to a 3D coordinate system;

rotating the point set into two dimensions of the 3D coordinate system by applying a rotation value to the point set;

identifying an observed contour comprising periphery points of the rotated point set;

aligning a model contour with the observed contour, the model contour comprising periphery points of an expected display medium;

inversely applying the rotation value to the aligned model contour to rotate the model contour into three dimensions of the 3D coordinate system;

evaluating a position and orientation of the display medium based on the aligned and rotated model contour; and

using the projector to project content onto the display medium at the evaluated position and orientation of the display medium.

2. The system of claim 1 , wherein the rotation value comprises a rotation matrix.

3. The system of claim 1 , the acts further comprising:

determining a surface normal of the observed display medium based at least in part on the point set; and

determining the rotation value based at least in part on the surface normal of the observed display medium.

4. The system of claim 1 , the acts further comprising:

determining a surface normal of the observed display medium based at least in part on the point set; and

determining the rotation value as a rotation matrix that rotates the surface normal of the observed display medium into the two dimensions of the 3D coordinate system.

5. The system of claim 1 , wherein the aligning comprises minimizing differences between the periphery points of the model contour and the periphery points of the observed contour.

6. The system of claim 1 , wherein the aligning comprises:

minimizing differences between the periphery points of the model contour and the periphery points of the observed contour in iterations; and

during successive iterations of the minimizing, ignoring periphery points that are least closely aligned.

7. The system of claim 1 , wherein the aligning comprises iterative closest point (ICP) minimization to minimize distances between the periphery points of the model contour and the periphery points of the observed contour in successive iterations, and wherein each iteration of the minimization ignores one or more periphery points that were least closely aligned in a previous iteration of the minimization.

8. The system of claim 1 , the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning.

9. The system of claim 1 , the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning, wherein the estimating is based at least in part on principal component analysis (PCA).

10. The system of claim 1 , the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning, wherein the estimating is based at least in part on averaging the periphery points of the observed contour to determine a center of the observed contour.

11. The system of claim 1 , wherein the display medium is rectangular.

12. A method of determining a pose of an object,

comprising:

obtaining a point set representing an observation of at least a portion of the object with respect to a 3D coordinate system;

rotating the point set into two dimensions of the 3D coordinate system by applying a rotation value to the point set;

aligning a model contour with a data contour, the model contour comprising periphery points corresponding to the object, the data contour comprising periphery points of the rotated point set; and

inversely applying the rotation value to the aligned model contour.

13. The method of claim 12 , wherein the rotation value comprises a rotation matrix.

14. The method of claim 12 , further comprising:

determining a surface normal of the observed display medium based at least in part on the point set; and

determining the rotation value based at least in part on the surface normal of the observed display medium.

15. The method of claim 12 , further comprising:

determining a surface normal of the observed display medium based at least in part on the point set; and

determining the rotation value as a rotation matrix that rotates the surface normal of the observed display medium into the two dimensions of the 3D coordinate system.

16. The method of claim 12 , wherein the aligning comprises minimizing differences between the periphery points of the model contour and the periphery points of the data contour.

17. The method of claim 12 , wherein the aligning comprises:

minimizing differences between the periphery points of the model contour and the periphery points of the data contour in iterations; and

during successive iterations of the minimizing, ignoring periphery points that are least closely aligned.

18. The method of claim 12 , wherein the aligning comprises iterative closest point (ICP) minimization to minimize distances between the periphery points of the model contour and the periphery points of the data contour in successive iterations, and wherein each iteration of the minimization ignores one or more periphery points that were least closely aligned in a previous iteration of the minimization.

19. The method of claim 12 , further comprising estimating alignment of the model contour with the data contour prior to performing the aligning, wherein the estimating is based at least in part on principal component analysis (PCA).

20. The method of claim 12 , further comprising estimating alignment of the model contour with the data contour prior to performing the aligning, wherein the estimating is based at least in part on averaging the periphery points of the data contour to determine a center of the data contour.

21. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

obtaining an observed point set representing at least a portion of an observed object in a 3D coordinate system;

rotating the observed point set into two dimensions of the 3D coordinate system;

creating an observed contour representing periphery points of the rotated observed point set;

creating a model contour representing periphery points of an expected object;

aligning the model contour with the observed contour; and

inversely rotating the aligned model contour into the 3D coordinate system.

22. The one or more non-transitory computer-readable media of claim 21 , wherein the 2D coordinate system comprises two dimensions of the 3D coordinate system.

23. The one or more non-transitory computer-readable media of claim 21 , wherein the aligning comprises minimizing differences between the periphery points of the model contour and the periphery points of the observed contour.

24. The one or more non-transitory computer-readable media of claim 21 , wherein the aligning comprises:

minimizing differences between the periphery points of the model contour and the periphery points of the observed contour in iterations; and

during successive iterations of the minimizing, ignoring periphery points that are least closely aligned.

25. The one or more non-transitory computer-readable media of claim 21 , wherein the aligning comprises iterative closest point minimization (ICP) to minimize distances between the periphery points of the model contour and the periphery points of the observed contour in successive iterations, and wherein each iteration of the minimization ignores one or more periphery points that were least closely aligned in a previous iteration of the minimization.

26. The one or more non-transitory computer-readable media of claim 21 , the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning, wherein the estimating is based at least in part on principal component analysis (PCA).

27. The one or more non-transitory computer-readable media of claim 21 , the acts further comprising estimating alignment of the model contour with the observed contour prior to performing the aligning, wherein the estimating is based at least in part on averaging the periphery points of the observed contour to determine a center of the observed contour.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 12, 2015
From: RAWLES LLC
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 037103/0084 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2012
From: LIU, QIANG; CHANG, SAMUEL HENRY
To: RAWLES LLC
Reel/Frame 028748/0701 →