IP Library Granted Patent US 11,151,394
Granted Patent B2
US 11,151,394 · App. 16/910,677 · Granted Oct 19, 2021

Identifying dynamic objects in a point cloud

Inventor: Derik Schroeter (Fremont, CA)
Assignee: NVIDIA CORPORATION
G06K9/00791G01C21/30G01S17/89G06K9/00523G06T7/10G06T7/50G06T7/70G06T2207/10028G06T2207/30252G08G1/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,151,394
App. No.
16/910,677
Granted
Oct 19, 2021
Kind
B2
Abstract

Operations may comprise obtaining a first point cloud from a map representing a region. The operations may also include obtaining a second point cloud from one or more sensors of a vehicle traveling through the region. In addition, the operations may include identifying one or more subsets of clusters of second points of the second point cloud. The operations may also include determining correspondences between first points of the first point cloud and cluster points of the one or more subsets of clusters of the second point cloud. Moreover, the operations may include identifying at least a cluster of the one or more subsets of clusters, the identified cluster having, with respect to first points of the first point cloud, a correspondence percentage that is less than a threshold value. The operations may also include adjusting the second point cloud based on the identified cluster.

Claims (53)

1. A computer-implemented method, comprising:

obtaining a first point cloud from a map representing a region;

obtaining a second point cloud from one or more sensors of a vehicle traveling through the region;

identifying one or more subsets of clusters of second points of the second point cloud;

determining correspondences between first points of the first point cloud and cluster points of the one or more subsets of clusters of the second point cloud;

identifying at least a cluster of the one or more subsets of clusters, the identified cluster having, with respect to first points of the first point cloud, a correspondence percentage that is less than a threshold value;

adjusting the second point cloud based on the identified cluster; and

determining a location of the vehicle in the region based on the adjusted second point cloud.

2. The method of claim 1 , further comprising assigning weights to the one or more clusters of second points in which the weights correspond to how useful the one or more clusters of second points are in determining which objects are dynamic.

3. The method of claim 1 , wherein identifying the identified cluster comprises:

performing segmentation of the second point cloud to identify segments of second points of the second point cloud in which the segments are identified based on the second points of a corresponding segment having a shared characteristic; and

organizing the segments into the one or more clusters of second points of the second point cloud.

4. The method of claim 1 , wherein the one or more clusters of second points of the second point cloud comprise a ground cluster associated with the ground, and wherein the ground cluster is given a lower priority than non-ground clusters with respect to determining correspondences between the first points and respective cluster points of the ground clusters and the non-ground clusters.

5. The method of claim 1 , further comprising selecting a sample of cluster points from the identified cluster, wherein determining the correspondences between the first points of the first point cloud and the cluster points of the identified cluster is only performed with respect to the selected sample of cluster points of the identified cluster.

6. The method of claim 1 , wherein adjusting the second point cloud comprises removing, from the second point cloud, the cluster points of the identified cluster in response to the identified cluster being determined as corresponding to a dynamic object.

7. The method of claim 1 , further comprising performing localization based on the adjusted second point cloud in which cluster points of the identified cluster are assigned less weight with respect to localization as compared to remaining points.

8. The method of claim 1 , wherein adjusting the second point cloud improves one or more of: localization of the vehicle, updating high definition map data of the region; or collision avoidance of a dynamic object by the vehicle.

9. A computer system comprising:

one or more processors; and

one or more non-transitory computer readable storage media storing instructions that in response to being executed by the one or more processors, cause the system to perform operations comprising:

obtaining a first point cloud from a map representing a region;

obtaining a second point cloud from one or more sensors of a vehicle traveling through the region;

identifying one or more subsets of clusters of second points of the second point cloud;

determining correspondences between first points of the first point cloud and cluster points of the one or more subsets of clusters of the second point cloud;

identifying at least a cluster of the one or more subsets of clusters, the identified cluster having, with respect to first points of the first point cloud, a correspondence percentage that is less than a threshold value;

adjusting the second point cloud based on the identified cluster; and

determining a location of the vehicle in the region based on the adjusted second point cloud.

10. The system of claim 9 , wherein the operations further comprise assigning weights to the one or more clusters of second points in which the weights correspond to how useful the one or more clusters of second points are in determining which objects are dynamic.

11. The system of claim 9 , wherein identifying the identified cluster comprises:

performing segmentation of the second point cloud to identify segments of second points of the second point cloud in which the segments are identified based on the second points of a corresponding segment having a shared characteristic; and

organizing the segments into the one or more clusters of second points of the second point cloud.

12. The system of claim 9 , wherein the one or more clusters of second points of the second point cloud comprise a ground cluster associated with the ground, and wherein the ground cluster is given a lower priority than non-ground clusters with respect to determining correspondences between the first points and respective cluster points of the ground clusters and the non-ground clusters.

13. The system of claim 9 , wherein the operations further comprise selecting a sample of cluster points from the identified cluster, wherein determining the correspondences between the first points of the first point cloud and the cluster points of the identified cluster is only performed with respect to the selected sample of cluster points of the identified cluster.

14. The system of claim 9 , wherein adjusting the second point cloud comprises removing, from the second point cloud, the cluster points of the identified cluster in response to the identified cluster being determined as corresponding to a dynamic object.

15. The system of claim 9 , wherein the operations further comprise performing localization based on the adjusted second point cloud in which cluster points of the identified cluster are assigned less weight with respect to localization as compared to remaining points.

16. The system of claim 9 , wherein adjusting the second point cloud improves one or more of: localization of the vehicle, updating high definition map data of the region; or collision avoidance of a dynamic object by the vehicle.

17. One or more non-transitory computer readable storage media storing instructions that in response to being executed by one or more processors, cause a system to perform operations comprising:

obtaining a first point cloud from a map representing a region;

obtaining a second point cloud from one or more sensors of a vehicle traveling through the region;

identifying one or more subsets of clusters of second points of the second point cloud;

determining correspondences between first points of the first point cloud and cluster points of the one or more subsets of clusters of the second point cloud;

identifying at least a cluster of the one or more subsets of clusters, the identified cluster having, with respect to first points of the first point cloud, a correspondence percentage that is less than a threshold value;

adjusting the second point cloud based on the identified cluster; and

determining a location of the vehicle in the region based on the adjusted second point cloud.

18. The one or more computer-readable storage media of claim 17 , wherein the operations further comprise assigning weights to the one or more clusters of second points in which the weights correspond to how useful the one or more clusters of second points are in determining which objects are dynamic.

19. The one or more computer-readable storage media of claim 17 , wherein identifying the identified cluster comprises:

performing segmentation of the second point cloud to identify segments of second points of the second point cloud in which the segments are identified based on the second points of a corresponding segment having a shared characteristic; and

organizing the segments into the one or more clusters of second points of the second point cloud.

20. The one or more computer-readable storage media of claim 17 , wherein the one or more clusters of second points of the second point cloud comprise a ground cluster associated with the ground, and wherein the ground cluster is given a lower priority than non-ground clusters with respect to determining correspondences between the first points and respective cluster points of the ground clusters and the non-ground clusters.

21. The one or more computer-readable storage media of claim 17 , wherein the operations further comprise selecting a sample of cluster points from the identified cluster, wherein determining the correspondences between the first points of the first point cloud and the cluster points of the identified cluster is only performed with respect to the selected sample of cluster points of the identified cluster.

22. The one or more computer-readable storage media of claim 17 , wherein adjusting the second point cloud comprises removing, from the second point cloud, the cluster points of the identified cluster in response to the identified cluster being determined as corresponding to a dynamic object.

23. The one or more computer-readable storage media of claim 17 , wherein the operations further comprise performing localization based on the adjusted second point cloud in which cluster points of the identified cluster are assigned less weight with respect to localization as compared to remaining points.

24. The one or more computer-readable storage media of claim 17 , wherein adjusting the second point cloud improves one or more of: localization of the vehicle, updating high definition map data of the region; or collision avoidance of a dynamic object by the vehicle.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2022
From: DEEPMAP INC.
To: NVIDIA CORPORATION
Reel/Frame 061038/0311 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 25, 2020
From: SCHROETER, DERIK
To: DEEPMAP INC.
Reel/Frame 053040/0139 →
Continuity (3)
Provisional Application 62865855 · Jun 24, 2019
Provisional Application 62866504 · Jun 25, 2019
Related Publication 20200401816A1 · Dec 24, 2020