IP Library Granted Patent US 9,846,946
Granted Patent B2
US 9,846,946 · App. 14/948,260 · Granted Dec 19, 2017

Objection recognition in a 3D scene

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,846,946
App. No.
14/948,260
Granted
Dec 19, 2017
Kind
B2
Abstract

A method comprising: obtaining a three-dimensional (3D) point cloud about at least one object of interest; detecting ground and/or building objects from 3D point cloud data using an unsupervised segmentation method; removing the ground and/or building objects from the 3D point cloud data; and detecting one or more vertical objects from the remaining 3D point cloud data using a supervised segmentation method.

Claims (93)

1. A method, comprising:

obtaining a three-dimensional (3D) point cloud about at least one object of interest;

detecting at least one of ground and building objects from 3D point cloud data using an unsupervised segmentation method;

removing the at least one of ground and building objects from the 3D point cloud data;

detecting one or more vertical objects from remaining 3D point cloud data using a supervised segmentation method;

dividing the 3D cloud point data into rectangular tiles in a horizontal plane; and

determining an estimation of a ground plane within the rectangular tiles, wherein determining the estimation of the ground plane within the rectangular tiles comprises:

dividing the rectangular tiles into a plurality of grid cells;

searching a minimal-z-value point within each grid cell;

searching points in the each grid cell having a z-value within a first predetermined threshold from the minimal-z-value of a grid cell;

collecting the points having the z-value within the first predetermined threshold from the minimal-z-value of the grid cell from each grid cell;

estimating the ground plane of a tile on the basis of the collected points; and

determining points locating within a second predetermined threshold from the estimated ground plane to comprise ground points of the tile.

2. The method according to claim 1 , further comprising:

projecting 3D points to range image pixels in a horizontal plane;

defining values of the range image pixels as a function of number of 3D points projected to the range image pixels and a maximal z-value among the 3D points projected to the range image pixels;

defining a geodesic elongation value for objects detected in the range image pixels; and

distinguishing buildings from other objects on the basis of the geodesic elongation value.

3. The method according to claim 2 , the method further comprising:

binarizing the values of the range image pixels;

applying morphological operations for merging neighboring points in the binarized values of the range image pixels; and

extracting contours for finding boundaries of the objects.

4. The method according to claim 1 further comprising applying a voxel based segmentation to the remaining 3D point cloud data, wherein the voxel based segmentation comprises:

performing voxilisation of the 3D point cloud data; and

merging of voxels into super-voxels and carrying out supervised classification based on discriminative features extracted from the super-voxels.

5. The method according to claim 4 further comprising:

merging the 3D points into voxels comprising a plurality of 3D points such that for a selected 3D point, all neighboring 3D points within a third predefined threshold from the selected 3D point are merged into a voxel without exceeding a maximum number of 3D points in the voxel.

6. The method according to claim 4 , further comprising:

merging any number of the voxels into the super-voxels such that a criteria is fulfilled, the criteria comprising:

a minimal geometrical distance between two voxels is smaller than a fourth predefined threshold; and

an angle between normal vectors of the two voxels is smaller than a fifth predefined threshold.

7. The method according to claim 4 further comprising:

extracting features from the super-voxels for classifying the super-voxels into objects.

8. The method according to claim 7 , wherein the features include one or more of:

geometrical shape;

height of a voxel above ground;

horizontal distance of the voxel to a center line of a street;

surface normal of the voxel;

voxel planarity;

density of 3D points in the voxel; and

intensity of the voxel.

9. An apparatus, comprising at least one processor, memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least:

obtain a three-dimensional (3D) point cloud about at least one object of interest;

detect at least one of ground and building objects from 3D point cloud data using an unsupervised segmentation method;

remove the at least one of ground and building objects from the 3D point cloud data; and

detect one or more vertical objects from remaining 3D point cloud data using a supervised segmentation method;

divide the 3D cloud point data into rectangular tiles in a horizontal plane; and

determine an estimation of a ground plane within the rectangular tiles, wherein to determine the estimation of the ground plane within the rectangular tiles, the apparatus is further caused to:

divide the rectangular tiles into a plurality of grid cells;

search a minimal-z-value point within each grid cell;

search points in the each grid cell having a z-value within a first predetermined threshold from the minimal-z-value of a grid cell;

collect the points having the z-value within the first predetermined threshold from the minimal-z-value of the grid cell from the each grid cell;

estimate the ground plane of a tile on the basis of the collected points; and

determine points locating within a second predetermined threshold from the estimated ground plane to comprise ground points of the tile.

10. The apparatus according claim 9 , wherein the apparatus is further caused to:

project 3D points to range image pixels in a horizontal plane;

define values of the range image pixels as a function of number of 3D points projected to the range image pixels and a maximal z-value among the 3D points projected to the range image pixels;

define a geodesic elongation value for objects detected in the range image pixels; and

distinguish buildings from other objects on the basis of the geodesic elongation value.

11. The apparatus according to claim 10 , wherein the apparatus is further caused to:

binarize the values of the range image pixels;

apply morphological operations for merging neighboring points in the binarized values of the range image pixels; and

extract contours for finding boundaries of the objects.

12. The apparatus according to claim 9 , wherein the apparatus is further caused to apply a voxel based segmentation to the remaining 3D point cloud data, and wherein to apply the voxel based segmentation, the apparatus is further caused to:

perform voxilisation of the 3D point cloud data;

merge voxels into super-voxels; and

carry out supervised classification based on discriminative features extracted from the super-voxels.

13. The apparatus according to claim 12 , wherein the apparatus is further caused to merge the 3D points into voxels comprising a plurality of 3D points such that for a selected 3D point, all neighboring 3D points within a third predefined threshold from the selected 3D point are merged into a voxel without exceeding a maximum number of 3D points in the voxel.

14. The apparatus according to claim 12 , wherein the apparatus is further caused to merging any number of the voxels into the super-voxels such that following criteria is fulfilled:

a minimal geometrical distance between two voxels is smaller than a fourth predefined threshold; and

an angle between normal vectors of the two voxels is smaller than a fifth predefined threshold.

15. The apparatus according to claim 12 , wherein the apparatus is the apparatus is further caused to extract features from the super-voxels for classifying the super-voxels into objects.

16. The apparatus according to claim 15 , wherein the features include one or more of:

geometrical shape;

height of a voxel above ground;

horizontal distance of the voxel to a center line of a street;

surface normal of the voxel;

voxel planarity;

density of 3D points in the voxel; and

intensity of the voxel.

17. A non-transitory computer readable storage medium stored with code thereon for use by an apparatus, which when executed by a processor, causes the apparatus to perform:

obtain a three-dimensional (3D) point cloud about at least one object of interest;

detect at least one of ground and building objects from 3D point cloud data using an unsupervised segmentation method;

remove the at least one of ground and building objects from the 3D point cloud data; and

detect one or more vertical objects from remaining 3D point cloud data using a supervised segmentation method;

divide the 3D cloud point data into rectangular tiles in a horizontal plane; and

determine an estimation of a ground plane within the rectangular tiles, wherein to determine the estimation of the ground plane within the rectangular tiles, the apparatus is further caused to perform:

divide the rectangular tiles into a plurality of grid cells;

search a minimal-z-value point within each grid cell;

search points in the each grid cell having a z-value within a first predetermined threshold from the minimal-z-value of a grid cell;

collect the points having the z-value within the first predetermined threshold from the minimal-z-value of the grid cell from the each grid cell;

estimate the ground plane of a tile on the basis of the collected points; and

determine points locating within a second predetermined threshold from the estimated ground plane to comprise ground points of the tile.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 23, 2020
From: NOKIA TECHNOLOGIES OY
To: VIVO MOBILE COMMUNICATION CO., LTD.
Reel/Frame 051595/0763 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 12, 2016
From: FAN, LIXIN; BABAHAJIANI, POURIA
To: NOKIA TECHNOLOGIES OY
Reel/Frame 037725/0511 →