IP Library › Granted Patent US 9,836,871
Granted Patent B2
US 9,836,871 · App. 14/419,204 · Granted Dec 5, 2017

Three-dimentional plane panorama creation through hough-based line detection

Inventor: Jaechoon Chon (Albany, CA)
Assignee: HERE Global B.V.
G06T15/08G06K9/4633G06T3/0037G06T3/0062G06T7/33G06T17/05G06T2200/08G06T2207/10028G06T2207/20061G06T2207/30184G06T2210/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,836,871
App. No.
14/419,204
Granted
Dec 5, 2017
Kind
B2
Abstract

A method for creating a plane panorama from point cloud data using Hough transformations is disclosed. The method involves converting the three-dimensional point cloud into a two-dimensional histogram with bins grouping neighboring points, and performing a Hough transformation on the histogram. The resulting transformed data is segmented and the method searches the segments iteratively for a major line, followed by lines that are orthogonal, diagonal, or parallel to the major line, and discards outlying data in each bin as lines are identified. The detected lines are connected to form planes, and the planes are assembled into a hole- and gap-filled panorama. The method may also use an algorithm such as a Random Sample Consensus (RANSAC) algorithm to detect a ground plane.

Claims (46)

1. A method of constructing a three-dimensional plane panorama, comprising:

detecting a ground plane in a three-dimensional point cloud;

converting the three-dimensional point cloud to a two-dimensional histogram with bins grouping neighboring points in the three-dimensional point cloud;

performing a Hough transformation on the two dimensional histogram to create a Hough accumulator matrix;

segmenting the Hough accumulator matrix into one or more segments, each segment including one or more of the bins;

iteratively performing line detection in each of the one or more segments;

discarding, as one or more lines are detected in the one or more segments, neighboring points in each of the bins that do not lie along the one or more lines;

discarding one or more lines from the line detection based on a threshold size, wherein the threshold size is a function of a distance from an origin; and

assembling the one or more lines into one or more planes.

2. The method of claim 1 , further comprising assembling the one or more planes into a plane panorama.

3. The method of claim 1 , further comprising performing one or both of gap or hole filling on the one or more planes.

4. The method of claim 1 , further comprising, prior to said converting, transforming the three-dimensional point cloud from a relative coordinate system to an absolute position coordinate system.

5. The method of claim 4 , wherein the absolute position coordinate system is established using Global Positioning System (GPS) data associated with the three-dimensional point cloud.

6. The method of claim 1 , wherein said detecting the ground plane uses a Random Sample Consensus (RANSAC) algorithm.

7. The method of claim 1 , wherein said iteratively performing line detection continues until line detection has been performed on all of the bins.

8. The method of claim 1 , wherein said iteratively performing line detection comprises:

detecting a first major line;

detecting one or more related lines that are orthogonal, parallel or diagonal with respect to the first major line;

choosing a largest one of the one or more related lines;

discarding outlying data from ones of the bins containing data related to the first major line and the one or more related lines; and

choosing a largest one of the one or more related lines as a next major line.

9. The method of claim 1 , wherein said converting the three-dimensional point cloud to the two-dimensional histogram further comprises discarding points in the three-dimensional point cloud that do not meet a predefined threshold.

10. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

detecting a ground plane in a three-dimensional point cloud;

converting the three-dimensional point cloud to a two-dimensional histogram with bins grouping neighboring points in the three-dimensional point cloud;

performing a Hough transformation on the two dimensional histogram to create a Hough accumulator matrix;

segmenting the Hough accumulator matrix into one or more segments, each segment including one or more of the bins;

iteratively performing line detection in each of the one or more segments;

discarding, as one or more lines are detected in the one or more segments, neighboring points in each of the bins that do not lie along the one or more lines;

discarding one or more lines from the line detection based on a threshold size, wherein the threshold size is a function of a distance from an origin; and

assembling the one or more lines into one or more planes.

11. The apparatus of claim 10 , the computer program code further configured to perform:

assembling the one or more planes into a panoramic image.

12. The apparatus of claim 10 , the computer program code further configured to perform:

performing gap filling on the one or more planes to complete a polyhedron representing a structure.

13. The apparatus of claim 10 , the computer program code further configured to perform:

transforming the three-dimensional point cloud from a relative coordinate system to an absolute position coordinate system.

14. The apparatus of claim 13 , wherein the absolute position coordinate system is established using Global Positioning System (GPS) data associated with the three-dimensional point cloud.

15. The apparatus of claim 10 , wherein iteratively performing line detection comprises:

detecting a first major line;

detecting one or more related lines that are orthogonal, parallel or diagonal with respect to the first major line;

choosing a largest one of the one or more related lines;

discarding outlying data from ones of the bins containing data related to the first major line and the one or more related lines; and

choosing a largest one of the one or more related lines as a next major line.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 21, 2015
From: EARTHMINE, INC.
To: HERE GLOBAL B.V.
Reel/Frame 035687/0287 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 3, 2015
From: CHON, JAECHOON
To: EARTHMINE, INC.
Reel/Frame 034872/0362 →
Continuity (2)
Provisional Application 61679006 · Aug 2, 2012
Related Publication 20150199839A1 · Jul 16, 2015