IP Library › Granted Patent US 9,514,574

Granted Patent B2

US 9,514,574 · App. 14/268,721 · Granted Dec 6, 2016

System and method for determining the extent of a plane in an augmented reality environment

Inventor: Gerhard Reitmayr (Vienna, AT)

Assignee: QUALCOMM Incorporated

G06T19/006 G06K9/00201 G06K9/00624 G06T7/0051 G06T15/00 G06T7/0057 G06T2207/10028

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 9,514,574
App. No.: 14/268,721
Filed: May 2, 2014
Granted: Dec 6, 2016
Kind: B2
Examiner: PERROMAT, CARLOS
Art Unit: 2611
USPC: 345/419

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Methods, systems, computer-readable media, and apparatuses for constructing a representation of a planar object are presented. In some embodiments, techniques for constructing a representation of a planar object are disclosed. According to some embodiments, a method for constructing a representation of a planar object may include obtaining a depth image of a physical scene as captured from a viewing position. The depth image may comprise a plurality of depth values and corresponding to a plurality of points in the physical scene. The method may further include identifying a planar surface along which the planar object is estimated to be positioned. Furthermore, the method may include constructing a support map. Moreover, the method may include constructing an occlusion map, the occlusion map indicating portions of the planar surface where the planar object is missing. Subsequently, the method may include constructing a representation of at least one boundary of the planar object, using the occlusion map.

Claims (94)

1. A method for constructing a representation of a planar object comprising:

obtaining a depth image of a physical scene as captured from a viewing position, the depth image comprising a plurality of depth values and corresponding to a plurality of points in the physical scene, each depth value indicating a distance from the viewing position to one of the plurality of points in the physical scene;

identifying a planar surface along which the planar object is estimated to be positioned;

constructing a support map, the support map indicating portions of the planar surface occupied by the planar object and possible surfaces of objects resting on top of the planar object;

constructing an occlusion map, the occlusion map indicating portions of the planar surface where the planar object is missing;

testing a boundary of the planar surface using the occlusion map and the support map;

constructing a boundary of the planar surface based on the testing;

constructing a representation of at least one boundary of the planar object, using the occlusion map and the support map;

constructing an occlusion gradient map based on the occlusion map and the at least one boundary of the planar object;

constructing a support gradient map based on the support map and the at least one boundary of the planar object; and

classifying a segment associated with a boundary of the planar object based on the occlusion gradient map and the support gradient map.

2. The method of claim 1 wherein the planar object is a table top in the physical scene.

3. The method of claim 1 wherein constructing the occlusion map comprises:

partitioning the planar surface into a plurality of cells; and

for each cell, counting a number of points having depth values large enough to extend from the viewing position and traverse the cell at the planar surface.

4. The method of claim 1 wherein constructing the support map comprises:

partitioning the planar surface into a plurality of cells; and

for each cell, counting a number of points having depth values as to extend from the viewing position to the cell or to a location within a vertical column of space above the cell.

5. The method of claim 1 wherein classifying the segment associated with the boundary of the planar object comprises:

projecting a line segment into the support map;

projecting the line segment into the occlusion map; and

determining whether the projected line segment coincidences with a true edge of the planar object.

6. The method of claim 1 wherein the depth image is obtained using a RGB depth camera.

7. A device for constructing a representation of a planar object comprising:

memory;

a camera to obtain a depth image of a physical scene as captured from a viewing position, the depth image comprising a plurality of depth values and corresponding to a plurality of points in the physical scene, each depth value indicating a distance from the viewing position to one of the plurality of points in the physical scene; and

one or more processors configured to:

identify a planar surface along which the planar object is estimated to be positioned;

construct a support map, the support map indicating portions of the planar surface occupied by the planar object and possible surfaces of objects resting on top of the planar object;

construct an occlusion map, the occlusion map indicating portions of the planar surface where the planar object is missing;

test a boundary of the planar surface using the occlusion map and the support map;

construct a boundary of the planar surface based on the testing;

construct a representation of at least one boundary of the planar object, using the occlusion map and the support map;

construct an occlusion gradient map based on the occlusion map and the at least one boundary of the planar object;

construct a support gradient map based on the support map and the at least one boundary of the planar object; and

classify a segment associated with a boundary of the planar object based on the occlusion gradient map and the support gradient map.

8. The device of claim 7 wherein the planar object is a table top in the physical scene.

9. The device of claim 7 wherein constructing the occlusion map comprises:

partitioning the planar surface into a plurality of cells; and

for each cell, counting a number of points having depth values large enough to extend from the viewing position and traverse the cell at the planar surface.

10. The device of claim 7 wherein constructing the support map comprises:

partitioning the planar surface into a plurality of cells; and

for each cell, counting a number of points having depth values as to extend from the viewing position to the cell or to a location within a vertical column of space above the cell.

11. The device of claim 7 wherein classifying the segment associated with the boundary of the planar object comprises:

projecting a line segment into the support map;

projecting the line segment into the occlusion map; and

determining whether the projected line segment coincidences with a true edge of the planar object.

12. The device of claim 7 wherein the camera is a RGB depth camera.

13. One or more non-transitory computer-readable media storing computer-executable instructions for constructing a representation of a planar object that, when executed, cause one or more computing devices included in a device to:

obtain a depth image of a physical scene as captured from a viewing position, the depth image comprising a plurality of depth values and corresponding to a plurality of points in the physical scene, each depth value indicating a distance from the viewing position to one of the plurality of points in the physical scene;